Dynamic Auto Sharding support for Scala Slick

This post is related to an issue already touched upon in Dynamically changing the database shard I am connecting to too .

Pointers to the code that should be changed to implement this feature are at https://github.com/slick/slick/issues/703

I'm new to Scala and Slick, can I get some help ?, on how to proceed with implementing this feature. Is there a slick / scala pattern to do this at the application level.

My problem is that "I have a connection pool of different MySQL shards, when I write a query / queries related to the id (roundabout keys), slick has to dynamically run that particular query on the corresponding database shard"

For example: if I write a request like this

  val q = for ( user <- users.filter(_.name === "cat")

                post <- posts.filter(_.postedBy === user.id)

                comment <- comments.filter(_.postId === post.id)

                } yield comment.content

    q.run

    a trivial case should be like one below.

    users += User(id = 1, name = "cat", email = "cat@mat.com") => hits shard no 1

      

Even if the user id, post id, comment id are dynamically generated, the stain has to be removed over the correct database tile using some sharing criteria (key (ID)% 3) and everything has to happen in the background like a single database query.

To implement a function at the application level

Is there a way to read the state of the Query object dynamically so that I can write a function like

 def func(q: Query[Something], shards: Map[Int, Database], num: Int): Unit = {

           shards(q.getId % num).withSession{ implicit session => {

              q.run

          }
 }

      

Using:

     val q = users.insert(User(id = 1, name = "cat", email = "cat@cat.com"))

     func(q, shards, 10) => q executes on one of the 10 shards.

      

Thank.

+3


source to share





All Articles