Slow SQLite performance on multi-user LAN

We are using SQLite as a shared DB in our application. (I know this is not the best solution, but server and client architecture is not possible) There are only a few users, a very small db and only a few entries.

The application is written in C # and we are using System.Data.SQLite.dll , but the problem also occurs, for example when using SQLiteDatabaseBrowser

As long as only one user connects to the DB and requests some results, this is very fast. Just a few milliseconds. One user can make multiple connections and execute selection commands in parallel. It also doesn't affect performance.

But as soon as another user from another machine connects to the db, the performance becomes very bad for every connected user. Performance doesn't work while all connections / apps are closed. After that, the first user connects, gets good performance until the next user connects.

I've tried a lot of things:

  • PRAGMA synchronous = OFF
  • updated to latest sqlite version (and created a new db file with this version)
  • DB-File read-only
  • share the network for everyone
  • connection string with various parameters (almost all)
  • various sqlite programs (our app and SQLiteDatabaseBrowser)
  • set of file systems (NTFS and FAT32)

After that, I wrote a small application that opens a connection, requests some results, and displays the elapsed time. It's all in an endless loop.

Here's the code for this simple app:

    static void Main(string[] args)
    {
        SQLiteConnectionStringBuilder conBuilder = new SQLiteConnectionStringBuilder();
        conBuilder.DataSource = args[0];
        conBuilder.Pooling = false;                                    
        conBuilder.ReadOnly = true;

        string connectionString = conBuilder.ConnectionString;

        while (true)
        {
            RunQueryInNewConnection(connectionString);
            System.Threading.Thread.Sleep(500);
        }
    }

    static void RunQuery(SQLiteConnection con)
    {
        using (SQLiteCommand cmd = con.CreateCommand())
        {
            cmd.CommandText = "select * from TabKatalog where ReferenzName like '%0%'";
            Console.WriteLine("Execute Query: " + cmd.CommandText);

            Stopwatch watch = new Stopwatch();
            watch.Start();

            int lines = 0;
            SQLiteDataReader reader = cmd.ExecuteReader();
            while (reader.Read())
                lines++;

            watch.Stop();
            Console.WriteLine("Query result: " + lines + " in " + watch.ElapsedMilliseconds + " ms");
        }
    }


    static void RunQueryInNewConnection(string pConnectionString)
    {
        using (SQLiteConnection con = new SQLiteConnection(pConnectionString, true))
        {
            con.Open();
            RunQuery(con);
        }

        System.Data.SQLite.SQLiteConnection.ClearAllPools();
        GC.Collect();
        GC.WaitForPendingFinalizers();           
    }

      

While testing with this little application, I realized that it was enough for another system to take the file descriptor into the sqlite db to slow down performance. It doesn't seem to have anything to do with dB. Performance is maintained until all file descriptors are freed. I traced it down with procexp.exe. In addition, only remote systems experience performance issues. On the host of the db file itself, queries are fast every time.

Anyone faced the same problem or had some clues?

+3


source to share


1 answer


Windows does not cache files that are simultaneously available on another computer.



If you need high concurrency, consider using a client / server database .

0


source







All Articles