How to: CRUD and JPA association handling using Kundera
by Vivek Mishra
Recently we have released Kundera-2.0.5. This post is all about demonstrating how to perform CRUD and association handling using kundera. Kundera is now enabled to use secondary index support provided by cassandra(0.7.x onwards), Hence this example will demonstrate how to leverage that benefit using same JPA style within in Kundera
Example i am referring can be found Here.
Run this script to create column family in cassandra with indexes: | => |
|
Entity: PersonCassandra.java
package com.impetus.kundera.examples.crud;import javax.persistence.Column; |
Configuration : Persistence.xml
<persistence xmlns=”http://java.sun.com/xml/ns/persistence”
xmlns:xsi=”http://www.w3.org/2001/XMLSchema-instance”
xsi:schemaLocation=”http://java.sun.com/xml/ns/persistence
https://raw.github.com/impetus-opensource/Kundera/Kundera-2.0.4/kundera-core/src/test/resources/META-INF/persistence_2_0.xsd”
version=”2.0″>
<!– Persistence Units for twissandra application –>
<persistence-unit name=”twissandra”>
<provider>com.impetus.kundera.KunderaPersistence</provider>
<properties>
<property name=”kundera.nodes” value=”localhost”/>
<property name=”kundera.port” value=”9160″/>
<property name=”kundera.keyspace” value=”KunderaExamples”/>
<property name=”kundera.dialect” value=”cassandra”/>
<property name=”kundera.client” value=”Pelops”/>
<property name=”kundera.cache.provider.class” value=”com.impetus.kundera.cache.ehcache.EhCacheProvider”/>
<property name=”kundera.cache.config.resource” value=”/ehcache-test.xml”/>
</properties>
</persistence-unit>
</persistence>
Now, if you notice @ table annotation:
@Table(name = “PERSON”, schema = “KunderaExamples@twissandra”)
For cassandra, “PERSON” is specified column family and schema denotes “keyspace@puname”.
Entity definition:
public class PersonTest |
Initialize entity manager
emf = Persistence.createEntityManagerFactory("twissandra"); |
Insert :
public void onInsertCassandra() |
Find By Id:
public void onFindByIdCassandra() |
Find By Name:
public void onFindByName() |
Find By Name and Age:
|
Find By Range:
public void onFindByRange() |
Find by Name and “<” and “>”
|
Self Association:
Example demonstrating about how to define and perform bi directional self association is available Here
Using Default lucene index
Still there are some operations not supported by enabling cassandra secondary indexes(e.g. indexing and search over super column values etc.). Also indexing support over HBase is not yet mature, so Kundera does provide default lucene indexing support for all sort of find operation. What you need to do is simply provide given below property:
<property name=”index_home_dir” value=”$LUCENE_DIR_PATH”/>
This will simply start storing and indexing records on specified local/remote location.
Can we use same example for other supported data stores(e.g. Mongo, HBase, Mysql etc) ?
Answer is YES. changes required:
- Define persistence unit in persistence.xml
- Create script specific to intended database.
- Modify entity definition (e.g. PersonCassandra) for correct column family name or table name.(see @table annotation above)
- Modify entity manager factory instantiation for correct column family name.
That’s it !
Hey Vivek,
How do I connect Kundera to an EmbeddedCassandraService.(org.apache.cassandra.service.EmbeddedCassandraService). Is it possible ? I want to use it in some unit tests.
Hi Faruk,
To connect/use kundera with EmbeddedCassandraService, you simply need to configure EmbeddedCassandraService for localhost and port. In earlier version of kundera we build some junits using EmbeddedCassandraService. Please find some links below for reference:
1. https://github.com/impetus-opensource/Kundera/blob/Kundera-2.0.1/src/test/java/com/impetus/kundera/junit/BaseTest.java (kundera link)
2. http://prettyprint.me/2010/02/14/running-cassandra-as-an-embedded-service/
(bit old, but useful)
3. http://www.massapi.com/class/em/EmbeddedCassandraService.html
-Vivek
Hi Vivek,
Thanks for the prompt reply. I can’t seem to get my Kundera entitymanager to connect to my embeddeds cassandra service. I started with the kundera 5 minute example. It works with my standalone cassandra server but when I try an embedded cassandra service it is not able to connect.
This is how I start my embedded cassandra service. Am I doing something wrong?
Thanks,
Faruk
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import org.apache.cassandra.io.util.FileUtils;
import org.apache.cassandra.config.DatabaseDescriptor;
import org.apache.thrift.transport.TTransportException;
import org.apache.cassandra.db.commitlog.CommitLog;
import org.apache.cassandra.utils.ByteBufferUtil;
import org.apache.cassandra.thrift.TBinaryProtocol;
import org.apache.thrift.transport.TFramedTransport;
import org.apache.cassandra.thrift.Compression;
import org.apache.cassandra.thrift.Cassandra;
import org.apache.thrift.transport.TSocket;
import org.apache.thrift.transport.TTransport;
import org.apache.thrift.protocol.TProtocol;
import org.apache.cassandra.service.EmbeddedCassandraService;
import org.apache.cassandra.config.ConfigurationException;
import org.apache.cassandra.contrib.utils.service.CassandraServiceDataCleaner;
public class TestCassandraServer {
private static final String TMP = “/tmp/cassandra-aware”;
private static final String yamlFile = “/cassandra.yaml”;
private EmbeddedCassandraService cassandra;
/**
* Set embedded cassandra up and spawn it in a new thread.
*
* @throws TTransportException
* @throws IOException
* @throws InterruptedException
*/
public void setup() throws TTransportException, IOException,
InterruptedException, ConfigurationException {
// delete tmp dir first
rmdir(TMP);
// make a tmp dir and copy cassandra.yaml and log4j.properties to it
copy(“/log4j-embedded-cassandra.properties”, TMP);
copy(yamlFile, TMP);
/* set system properties for cassandra */
System.setProperty(“cassandra.config”, “file:” + TMP + yamlFile);
System.setProperty(“log4j.configuration”, “file:” + TMP
+ “/log4j-embedded-cassandra.properties”);
System.setProperty(“cassandra-foreground”, “true”);
cleanupAndLeaveDirs();
CassandraServiceDataCleaner cleaner = new CassandraServiceDataCleaner();
cleaner.prepare();
cassandra = new EmbeddedCassandraService();
cassandra.start();
System.out.println(“EmbeddedCassandraService Started”);
loadTables();
}
public void teardown() {
dropKeyspaces();
}
public static void cleanEmbeddedCassandra() {
dropKeyspaces();
}
private static void dropKeyspaces() {
Cassandra.Client client;
try {
client = getClient();
String query = “DROP KEYSPACE KunderaKeyspace”;
client.execute_cql_query(ByteBufferUtil.bytes(query), Compression.NONE);
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
private void loadTables(){
Cassandra.Client client;
try {
client = getClient();
String query = “CREATE KEYSPACE KunderaKeyspace WITH strategy_options:replication_factor=1 AND strategy_class = ‘SimpleStrategy'”;
client.execute_cql_query(ByteBufferUtil.bytes(query), Compression.NONE);
client.set_keyspace(“KunderaKeyspace”);
String cf = “create column family users with comparator=UTF8Type and default_validation_class=UTF8Type and key_validation_class=UTF8Type”;
client.execute_cql_query(ByteBufferUtil.bytes(cf), Compression.NONE);
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
private static void rmdir(String dir) throws IOException {
File dirFile = new File(dir);
if (dirFile.exists()) {
FileUtils.deleteRecursive(new File(dir));
}
}
/**
* Copies a resource from within the jar to a directory.
*
* @param resource
* @param directory
* @throws IOException
*/
private static void copy(String resource, String directory) throws IOException {
mkdir(directory);
InputStream is = TestCassandraServer.class.getResourceAsStream(resource);
String fileName = resource.substring(resource.lastIndexOf(“/”) + 1);
File file = new File(directory + System.getProperty(“file.separator”) + fileName);
OutputStream out = new FileOutputStream(file);
byte buf[] = new byte[1024];
int len;
while ((len = is.read(buf)) > 0) {
out.write(buf, 0, len);
}
out.close();
is.close();
}
/**
* Creates a directory
*
* @param dir
* @throws IOException
*/
private static void mkdir(String dir) throws IOException {
FileUtils.createDirectory(dir);
}
private static void cleanupAndLeaveDirs() throws IOException {
mkdirs();
cleanup();
mkdirs();
CommitLog.instance.resetUnsafe(); // cleanup screws w/ CommitLog, this
// brings it back to safe state
}
private static void cleanup() throws IOException {
// clean up commitlog
String[] directoryNames = { DatabaseDescriptor.getCommitLogLocation(), };
for (String dirName : directoryNames) {
File dir = new File(dirName);
if (!dir.exists())
throw new RuntimeException(“No such directory: ”
+ dir.getAbsolutePath());
FileUtils.deleteRecursive(dir);
}
// clean up data directory which are stored as data directory/table/data
// files
for (String dirName : DatabaseDescriptor.getAllDataFileLocations()) {
File dir = new File(dirName);
if (!dir.exists())
throw new RuntimeException(“No such directory: ”
+ dir.getAbsolutePath());
FileUtils.deleteRecursive(dir);
}
}
public static void mkdirs() {
try {
DatabaseDescriptor.createAllDirectories();
} catch (IOException e) {
throw new RuntimeException(e);
}
}
public static Cassandra.Client getClient() throws Exception {
TTransport transport;
TProtocol proto;
TSocket socket;
Cassandra.Client client;
socket = new TSocket(“localhost”, 9160);
transport = new TFramedTransport(socket);
proto = new TBinaryProtocol(transport);
transport.open();
client = new Cassandra.Client(proto);
return client;
}
}
Is your server getting started? I hope you must have tried connecting to it with thrift client.
If CassandraDaemon is started , it should get connected.
Could you please paste error you are getting?
-Vivek
Hi Vivek,
I figured it out. Here is the class that works.
Thanks again,
Faruk
import java.io.IOException;
import org.apache.thrift.transport.TTransportException;
import org.apache.cassandra.utils.ByteBufferUtil;
import org.apache.cassandra.thrift.TBinaryProtocol;
import org.apache.thrift.transport.TFramedTransport;
import org.apache.cassandra.thrift.Compression;
import org.apache.cassandra.thrift.Cassandra;
import org.apache.thrift.transport.TSocket;
import org.apache.thrift.transport.TTransport;
import org.apache.thrift.protocol.TProtocol;
import org.apache.cassandra.service.EmbeddedCassandraService;
import org.apache.cassandra.config.ConfigurationException;
import org.apache.cassandra.contrib.utils.service.CassandraServiceDataCleaner;
public class TestCassandraServer {
private EmbeddedCassandraService cassandra;
/**
* Set embedded cassandra up and spawn it in a new thread.
*
* @throws TTransportException
* @throws IOException
* @throws InterruptedException
*/
public void setup() throws TTransportException, IOException,
InterruptedException, ConfigurationException {
CassandraServiceDataCleaner cleaner = new CassandraServiceDataCleaner();
cleaner.prepare();
cassandra = new EmbeddedCassandraService();
cassandra.start();
System.out.println(“EmbeddedCassandraService Started”);
loadTables();
}
public void teardown() {
dropKeyspaces();
}
public static void cleanEmbeddedCassandra() {
dropKeyspaces();
}
private static void dropKeyspaces() {
Cassandra.Client client;
try {
client = getClient();
String query = “DROP KEYSPACE DrDslKeyspace”;
client.execute_cql_query(ByteBufferUtil.bytes(query), Compression.NONE);
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
private void loadTables(){
Cassandra.Client client;
try {
client = getClient();
String query = “CREATE KEYSPACE DrDslKeyspace WITH strategy_options:replication_factor=1 AND strategy_class = ‘SimpleStrategy'”;
client.execute_cql_query(ByteBufferUtil.bytes(query), Compression.NONE);
client.set_keyspace(“DrDslKeyspace”);
String cf = “create columnfamily users”
+ ” (key text primary key) ”
+ “with comparator = ‘UTF8Type’ AND default_validation = ‘UTF8Type'”;
client.execute_cql_query(ByteBufferUtil.bytes(cf), Compression.NONE);
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
public static Cassandra.Client getClient() throws Exception {
TTransport transport;
TProtocol proto;
TSocket socket;
Cassandra.Client client;
socket = new TSocket(“127.0.0.1”, 9160);
transport = new TFramedTransport(socket);
proto = new TBinaryProtocol(transport);
transport.open();
client = new Cassandra.Client(proto);
return client;
}
}
Hi Vivek,
I want to use the query “select u from User u where u.privacyLevel=privacyLevel2”. I am not getting the output. It is saying Index column with EQ operator is not there. I am using OPP, Not added index_dir_home in persistance.xml file. If i am using “select u from User u” i will get all the objects. Can you tell me the configurations that i have to change, I am using Kundera 2.0.5.
Thanks,
Vineeth
As far as OPP is concerned, support for range query matters. Rest if privacyLevel is not a row key but it is indexed, it should work!
Looking at your query:
“select u from User u where u.privacyLevel=privacyLevel2″ . If it is a row key then it must be utf8 encoded in schema, if it is not row key, it is simple query over indexed key and should work.
I hope you have created script mentioned on top of blog to replicate blog post (or on User column family).
secondry indexes are not supported over super column family by cassandra.
As per error: ” It is saying Index column with EQ operator is not there”
it looks like column privacyLevel is not indexed?
Cheers,
Vivek
Thanks Vivek,
I tried with OPP and got the output.
If i want to use the random partition and lucene index, what changes i have to make in kundera configuration apart from adding the index_dir_home in persistance.xml.
One more thing, if i am using the lucene index, do i have to specify the index in the create column family script?
Apart from specifying “index_home_dir” nothing else is required. Currently kundera support index over all columns. Use of “selective index” will be release in 2.0.6.
-Vive
Hi vivek,
This is my create column family script,
“create column family user with comparator=UTF8Type and default_validation_class=UTF8Type and key_validation_class=UTF8Type;”
Entry in the persistance.xml
“”
Should i make any modification in LuceneIndexer,java.
Currently it’s given as index = new RAMDirectory();
Should i change it to index = FSDirectory.open(getIndexDirectory());
If it is FSDirectory, it will create three empty files in the folder E:\HIN_FILES\Lucene.
If it is Ram directory it wont create.
Thanks,
Vineeth.
persistance.xml
Once you close your emf. those indexes will be flushed to your FS. or even you fire a read, it will flush them out. No need to change.
RAMDirectory is pointing to configured index directory(e.g. index_home_dir).
Make sure to invoke emf.close() to flush them down to your FS.
-Vivek
Hi, I tried your example above with play framework. But I got this error. Could you please tell me why? [PersistenceLoaderException: com.impetus.kundera.utils.InvalidConfigurationException: org.xml.sax.SAXParseException: Open quote is expected for attribute “{1}” associated with an element type “xmlns”.]
Hi,
This example is bit old and you may want to refer:
https://github.com/impetus-opensource/Kundera/blob/trunk/kundera-cassandra/src/test/java/com/impetus/client/crud/PersonCassandraTest.java
In case still does not work for you, Please share more on error logs.
Hi mevivs, I am sorry , maybe my questions must be little silly. Because I have never worked with any noSQL DB. That is why I am struggling with integrating cassandra with java play framework. I decided to use kundera. I would like to show you my code i wrote so far. I just wanted to connect cassandra and do some crud operation from java play framework over kundera. So I thought that would be usefull for me.
This is what i have done so far.. Please show me where my mistake is .
Firstly I create keyspace and one column family in cassandra-cli
>CREATE KEYSPACE KunderaExamples;
>create column family PERSON with comparator=UTF8Type and column_metadata=[{column_name: PERSON_NAME, validation_class:UTF8Type, index_type: KEYS}];
And this was ok.
Then I created an play2.0 java app. and added kundera-cassandra-2.2.1-jar-with-dependencies.jar to the classpath and created META-INF folder containing persistence.xml in conf folder. This is my persistence.xml
com.impetus.kundera.KunderaPersistence
And I add this to application.conf jpa.default=twissandra
Then I defined a class in models package.
package models;
import javax.persistence.Column;
import javax.persistence.Entity;
import javax.persistence.Id;
import javax.persistence.Table;
@Entity
@Table(name = “PERSON”, schema = “KunderaExamples@twissandra”)
public class PersonCassandra{/** The person id. */
@Id
@Column(name = “PERSON_ID”)
private String personId;
@Column(name = “PERSON_NAME”)
private String personName;
@Column(name = “AGE”)
private Integer age;
public String getPersonId() {
return personId;
}
public void setPersonId(String personId) {
this.personId = personId;
}
public String getPersonName() {
return personName;
}
public void setPersonName(String personName) {
this.personName = personName;
}
public Integer getAge() {
return age;
}
public void setAge(Integer age) {
this.age = age;
}
}
And then my controller class.
package controllers;
import javax.persistence.EntityManager;
import javax.persistence.EntityManagerFactory;
import javax.persistence.Persistence;
import models.PersonCassandra;
import play.*;
import play.db.jpa.JPA;
import play.mvc.*;
import views.html.*;
public class Application extends Controller {
public static Result index() {
EntityManager em = JPA.em();
PersonCassandra o = new PersonCassandra();
o.setPersonId(“1”);
o.setPersonName(“XXXX YYYY”);
o.setAge(20);
em.persist(o);
return ok(index.render(“Your new application is ready.”));
}
}
That is what i have done so far.. And i got several exception. Let me share with you.
PersistenceLoaderException: com.impetus.kundera.utils.InvalidConfigurationException: org.xml.sax.SAXParseException: Element type “persistence” must be followed by either attribute specifications, “>” or “/>”.
[KunderaException: java.lang.IllegalArgumentException: Entity object is invalid, operation failed. Please check previous log message for details]
Hi,
lease have a look at https://github.com/impetus-opensource/Kundera/wiki/Getting-Started-in-5-minutes if you starting with Kundera. Else please email of share your drop box link to look into project.
This is the project implemented in play2.0, cassandra1.1.9;
https://www.dropbox.com/s/61qda5ib351wosg/SOW5.rar
Sincerely,
Emin
I also tried the example ‘Getting started in minutes’ on githup. This exception
[PersistenceLoaderException: com.impetus.kundera.utils.InvalidConfigurationException: Duplicate persistence-units for name: cassandra_pu. verify your persistence.xml file]
Try changing pu name in your persistence.xml from “cassandra_pu” to some other name say “mycassandrapu” and also change the same in your entity definition. Reason is, you are using jar-with-dependencies which is available in classpath. Jar-with-dependencies is containing some persistence.xml in classpath with same persistence unit name “cassandra_pu” .
I would suggest you to use pom/ant based model for application and add kundera-cassandra as maven dependency to be downloaded. Jar-with-dependencies is only recommended for small POC application.
Really thank you mevivs. I will try it. Do you have anything about what you suggess me so that it can guide me?
Hi,
You can have a look at kundera-tests package and test cases in kundera github site for more details.
I would request you to join kundera-discuss group for more help from community.
-Vivek
Reblogged this on Anand Kumar Singh.