Spark Cassandra Connector Filter With IN Clause
I am facing some issues with filtering spark cassandra connector for java. Cassandra allows you to filter the last key column of a section with an IN clause. eg
create table cf_text
(a varchar,b varchar,c varchar, primary key((a,b),c))
Query : select * from cf_text where a ='asdf' and b in ('af','sd');
sc.cassandraTable("test", "cf_text").where("a = ?", "af").toArray.foreach(println)
How do I count to the IN clause that is used in a spark mode CQL query? How do I specify range queries?
source to share
Just wondering, but does your Spark code above work? I thought Spark would not resolve WHERE
on partition keys ( a
and b
in your case) as it uses them under the hood (see the last answer to this question): Spark Datastax Java API Select Assertions
Anyway, with the Cassandra Spark connector, you are allowed to add your sentences WHERE
, or IN
you can specify with List<String>
.
List<String> valuesList = new ArrayList<String>();
valuesList.Add("value2");
valuesList.Add("value3");
sc.cassandraTable("test", "cf")
.where("column1 = ?", "value1")
.where("column2 IN ?", valuesList)
.keyBy(new Function<MyCFClass, String>() {
public String call(MyCFClass _myCF) throws Exception {
return _myCF.getId();
}
});
Note that the normal rules for using IN with Cassandra / CQL still apply here.
Range queries are specified in a similar way:
sc.cassandraTable("test", "person")
.where("age > ?", "15")
.where("age < ?", "20")
.keyBy(new Function<Person, String>() {
public String call(Person _person) throws Exception {
return _person.getPersonid();
}
});
source to share