How to efficiently retrieve data from a C file

I have a program that needs to extract (on startup) data from a text file. This file can get huge and I was wondering how I can speed up the process and evaluate its current performance. The code used to retrieve the data is as follows:

void startUpBillsLoading(Bill *Bills)
{
    FILE *BillsDb = 0, *WorkersDb = 0, *PaymentDb = 0;
    BillsDb = fopen("data/bills.db", "r");
    WorkersDb = fopen("data/workers.db", "r");
    PaymentDb = fopen ("data/payments.db", "r");
    char *Buffer = malloc (512);

    if (BillsDb && WorkersDb && PaymentsDb)
    {
        int i = 0, j = 0;

        while (fscanf (BillsDb, "%d;%[^;];%[^;];%[^;];%[^;];%d/%d/%d;%d/%d/%d;%d;%f;%f\n",
                &Bills[i].Id,
                Bills[i].CompanyName,
                Bills[i].ClientName,
                Bills[i].DepartureAddress,
                Bills[i].ShippingAddress,
                &Bills[i].Creation.Day,
                &Bills[i].Creation.Month,
                &Bills[i].Creation.Year,
                &Bills[i].Payment.Day,
                &Bills[i].Payment.Month,
                &Bills[i].Payment.Year,
                &Bills[i].NumWorkers,
                &Bills[i].TotalHT,
                &Bills[i].Charges) == 14)
        {
            Bills[i].Workers = 
                malloc (sizeof(Employee)*Bills[i].NumWorkers);

            fscanf (PaymentDb, "%d;%d;%[^;];%[^;];%[^\n]\n",
                    &Bills[i].Id,
                    &Bills[i].PaymentDetails.Method,
                    Bills[i].PaymentDetails.CheckNumber,
                    Bills[i].PaymentDetails.VirementNumber,
                    Bills[i].PaymentDetails.BankName);

            LatestBillId++;
            i++;
        }

        i = 0;
        while (fscanf (WorkersDb, "%d;%[^;];%[^;];%f\n",
                    &Bills[i].Id,   
                    Bills[i].Workers[j].Surname,
                    Bills[i].Workers[j].Name,
                    &Bills[i].Workers[j].Salary) == 4)
        {
            for (int j = 1; j <= Bills[i].NumWorkers-1; j++)
            {
                fscanf (WorkersDb, "%d;%[^;];%[^;];%f\n",
                                &Bills[i].Id,   
                                Bills[i].Workers[j].Surname,
                                Bills[i].Workers[j].Name,
                                &Bills[i].Workers[j].Salary);
            }
            i++;
        }

        fclose(BillsDb);
        fclose(WorkersDb);
        fclose(PaymentDb);
    }
    else
        printf ("\t\t\tImpossible d'acceder aux factures !\n");

    free (Buffer);
}

      

I used a library time.h

to measure the time it takes to get all the data I need. The ticket data is split into 3 files: bills.db, workers.db and payments.db. Each line of the file from bills.db

and from payments.db

represents a complete invoice, whereas the workers.db

number of lines required to represent an invoice is variable and depends on the number of employees associated with the bill.

I created these 3 files this way:

  • bills.db

    and payments.db

    has 118087 lines (thus how many accounts)
  • Each account was set (arbitrarily) to have 4 workers, so the file workers.db

    has 118087 * 4 = 472348 lines.

The time it takes for this function to run fully is about 0.9 seconds. How good (or bad) is this time and how to improve it?

+3


source to share


1 answer


There are few things you should read. The first is asymptotic time complexity and asymptotic space complexity , and the second is Big O notation . Big O notation tells how well a program works. For the code you provided, the Big O complexity is O (n ^ 2) aprox. Therefore the maximum limit is good, as it is the same as for quick sort, but since the data you are using is long, the load times will always be added to your runtime. If you want to improve, try to minimize the length of your data and read the smallest amount from this file. since if the value of n increases, the time will increase rapidly. you can readasymptotic notation and Big O sign from here



0


source







All Articles