External Value Storages
1 Introduction
By default JCR Values are stored in the Workspace Data container along with the JCR structure (i.e. Nodes and Properties). eXo JCR offers an additional option of storing JCR Values separately from Workspace Data container, which can be extremely helpful to keep Binary Large Objects (BLOBs) for example (see
Binary values processing).
Value storage configuration is a part of
Repository configuration, find more details there.
Tree-based storage is recommended for most of cases. If you run an application on Amazon EC2 - the S3 option may be interesting for architecture. Simple 'flat' storage is good in speed of creation/deletion of values, it might be a compromise for a small storages.
2 Tree File Value Storage
Holds Values in tree-like FileSystem files.
path property points to the root directory to store the files.
This is a recommended type of external storage, it can contain large amount of files limited only by disk/volume free space.
A disadvantage it's a higher time on Value deletion due to unused tree-nodes remove.
<value-storage id="Storage #1" class="org.exoplatform.services.jcr.impl.storage.value.fs.TreeFileValueStorage">
<properties>
<property name="path" value="data/values"/>
</properties>
<filters>
<filter property-type="Binary" min-value-size="1M"/>
</filters>
Where :
id - the value storage unique identifier, used for linking with properties stored in workspace container
path - a location where value files will be stored
Each file value storage can have the
filter(s) for incoming values. A filter can match values by property type (
property-type), property name (
property-name), ancestor path (
ancestor-path) and/or size of values stored (
min-value-size, in bytes).
In code sample we use a filter with
property-type and
min-value-size only. I.e. storage for binary values with size greater of 1MB.
It's recommended to store properties with large values in file value storage only.
Another example shows a value storage with different locations for large files (
min-value-size a 20Mb-sized filter). A value storage uses ORed logic in the process of filter selection. That means the first filter in the list will be asked first and if not matched the next will be called etc. Here a value matches the 20 MB-sized filter
min-value-size and will be stored in the path "data/20Mvalues", all other in "data/values".
<value-storages>
<value-storage id="Storage #1" class="org.exoplatform.services.jcr.impl.storage.value.fs.TreeFileValueStorage">
<properties>
<property name="path" value="data/20Mvalues"/>
</properties>
<filters>
<filter property-type="Binary" min-value-size="20M"/>
</filters>
<value-storage>
<value-storage id="Storage #2" class="org.exoplatform.services.jcr.impl.storage.value.fs.TreeFileValueStorage">
<properties>
<property name="path" value="data/values"/>
</properties>
<filters>
<filter property-type="Binary" min-value-size="1M"/>
</filters>
<value-storage>
<value-storages>3 S3 File Value Storage
Holds Values at Amazon S3 storage. For more about S3 see
http://www.amazon.com/S3-AWS-home-page-Money/b/ref=sc_fe_l_2/103-7720231-3235021?ie=UTF8&node=16427261&no=3435361&me=A36L942TSJ2AJA.
This type of storage saves all matching Values on Amazon S3 service. That is very useful for
cloud computing (like Amazon EC2 hosted repositories).
But can be used in any environment and storages combinations. It's often used in combined with
Workspace Simple DB storage.
It's networked storage with RESTbased access (via HTTP) that makes a footprint on performance of the Repository.
<value-storage id="Storage #1" class="org.exoplatform.services.jcr.impl.storage.value.s3.SimpleS3ValueStorage">
<properties>
<property name="bucket" value="BUCKET NAME HERE"/>
<property name="aws-access-key" value="INSERT YOUR AWS ACCESS KEY ID HERE"/>
<property name="aws-secret-access-key" value="INSERT YOUR AWS SECRET ACCESS KEY HERE"/>
<property name="s3-swap-directory" value="s3swap_directory_name"/>
</properties>
<filters>
<filter property-type="Binary"/>
</filters>
</value-storage>4 Simple File Value Storage
Not recommended to use in production due to low capacity capabilities on most file systems
But if you're sure in your file-system or data amount is small it may be useful for you as haves a faster speed of Value removal
Holds Values in flat FileSystem files.
path property points to root directory in order to store files
<value-storage id="Storage #1" class="org.exoplatform.services.jcr.impl.storage.value.fs.SimpleFileValueStorage">
<properties>
<property name="path" value="data/values"/>
</properties>
<filters>
<filter property-type="Binary" min-value-size="1M"/>
</filters>5 Content Addressable Value storage (CAS) support
Available from version 1.9.3
eXo JCR supports
Content-addressable storage feature for
Values storing.
Content-addressable storage, also referred to as associative storage and abbreviated CAS, is a mechanism for storing information that can be retrieved based on its content, not its storage location. It is typically used for high-speed storage and retrieval of fixed content, such as documents stored for compliance with government regulations.
Content Addressable Value storage stores unique content
once. Different properties (values) with same content will be stored as one data file shared between those values. We can tell the
Value content will be shared across some
Values in storage and will be stored on one physical file.
Storage size will be decreased for application which governs potentially same data in the content.
For example: if you have 100 different properties containing the same data (e.g. mail attachment) the storage stores only one single file. The file will be shared with all referencing properties.
If property
Value changes it is stored in an additional file. Alternatively the file is shared with other values, pointing to the same content.
The storage calculates
Value content address each time the property was changed. CAS write operations are much more expensive compared to the non-CAS storages.
Content address calculation based on
java.security.MessageDigest hash computation and tested with
MD5 and
SHA1 algorithms.
CAS storage works most efficiently on data that does not change often. For data that changes frequently, CAS is not as efficient as location-based addressing
CAS support can be enabled for
Tree and
Simple File Value Storage types.
To enable CAS support just configure it in JCR Repositories configuration like we do for other Value Storages.
<workspaces>
<workspace name="ws">
<container class="org.exoplatform.services.jcr.impl.storage.jdbc.JDBCWorkspaceDataContainer">
<properties>
<property name="source-name" value="jdbcjcr"/>
<property name="dialect" value="oracle"/>
<property name="multi-db" value="false"/>
<property name="update-storage" value="false"/>
<property name="max-buffer-size" value="200k"/>
<property name="swap-directory" value="target/temp/swap/ws"/>
</properties>
<value-storages>
<!------------------- here ----------------------->
<value-storage id="ws" class="org.exoplatform.services.jcr.impl.storage.value.fs.CASableTreeFileValueStorage">
<properties>
<property name="path" value="target/temp/values/ws"/>
<property name="digest-algo" value="MD5"/>
<property name="vcas-type" value="org.exoplatform.services.jcr.impl.storage.value.cas.JDBCValueContentAddressStorageImpl"/>
<property name="jdbc-source-name" value="jdbcjcr"/>
<property name="jdbc-dialect" value="oracle"/>
</properties>
<filters>
<filter property-type="Binary"/>
</filters>
</value-storage>
</value-storages>
Properties:
digest-algo - digest hash algorithm (MD5 and SHA1 were tested)
vcas-type - Value CAS internal data type, JDBC backed is currently implemented org.exoplatform.services.jcr.impl.storage.value.cas.JDBCValueContentAddressStorageImpl
jdbc-source-name - JDBCValueContentAddressStorageImpl specific parameter, database will be used to save CAS metadata. It's simple to use same as in workspace container.
jdbc-dialect - JDBCValueContentAddressStorageImpl specific parameter, database dialect. It's simple to use same as in workspace container.
