public class MemcachedStorageService extends net.shibboleth.utilities.java.support.component.AbstractIdentifiableInitializableComponent implements StorageService
https://code.google.com/p/memcached/wiki/NewProgrammingTricks#Namespacing
This storage service supports arbitrary-length context names and keys despite the 250-byte limit on memcached keys. Keys whose length is greater than 250 bytes are hashed using the SHA-512 algorithm and hex encoded to produce a 128-character key that is stored in memcached. Collisions are avoided irrespective of hashing by using the memcached add operation on all create operations which guarantees that an entry is created if and only if a key of the same value does not already exist. Note that context names and keys are assumed to have single-byte encodings in UTF-8 (i.e. ASCII characters) such that key lengths are equal to their size in bytes. Hashed keys naturally meet this requirement.
An optional context key-tracking feature is available to support updateContextExpiration(String, Long).
Key tracking is disabled by default, but can be enabled by setting the enableContextKeyTracking
parameter in the MemcachedStorageService(net.spy.memcached.MemcachedClient, int, boolean) constructor.
While there is modest performance impact for create and delete operations, the feature limits the number of keys
per context. With the default 1M memcached slab size, in the worst case 4180 keys are permitted per context.
In many if not most situations the value is easily double that. The limitation can be overcome by increasing the
slab size, which decreases overall cache memory consumption efficiency. When key tracking is disabled, there is no
limit on the number of keys per context other than overall cache capacity.
Limitations and requirements
deleteWithVersion(long, String, String) and deleteWithVersion(long, Object)
will throw runtime errors under the ASCII protocol.| Modifier and Type | Field and Description |
|---|---|
private MemcachedStorageCapabilities |
capabilities
Invariant storage capabilities.
|
private net.spy.memcached.MemcachedClient |
client
Memcached client instance.
|
protected static java.lang.String |
CTX_KEY_BLACKLIST_SUFFIX
Key suffix for entry that contains a list of blacklisted (deleted) context keys.
|
private static java.lang.String |
CTX_KEY_LIST_DELIMITER
Delimiter of items in the context key list.
|
protected static java.lang.String |
CTX_KEY_LIST_SUFFIX
Key suffix for entry that contains a list of context keys.
|
private org.slf4j.Logger |
logger
Logger instance.
|
private static int |
MAX_KEY_LENGTH
Maximum length in bytes of memcached keys.
|
private net.spy.memcached.transcoders.Transcoder<MemcachedStorageRecord> |
storageRecordTranscoder
Handles conversion of
MemcachedStorageRecord to bytes and vice versa. |
private net.spy.memcached.transcoders.Transcoder<java.lang.String> |
stringTranscoder
Handles conversion of strings to bytes and vice versa.
|
private int |
timeout
Memcached asynchronous operation timeout in seconds.
|
private boolean |
trackContextKeys
Flag that controls context key tracking.
|
| Constructor and Description |
|---|
MemcachedStorageService(net.spy.memcached.MemcachedClient client,
int timeout)
Creates a new instance.
|
MemcachedStorageService(net.spy.memcached.MemcachedClient client,
int timeout,
boolean enableContextKeyTracking)
Creates a new instance with optional context key tracking.
|
| Modifier and Type | Method and Description |
|---|---|
boolean |
create(java.lang.Object value)
Creates a new record in the store using an annotated object as the source.
|
boolean |
create(java.lang.String context,
java.lang.String key,
java.lang.Object value,
StorageSerializer serializer,
java.lang.Long expiration)
Creates a new record in the store with an expiration, using a custom serialization
process for an arbitrary object.
|
boolean |
create(java.lang.String context,
java.lang.String key,
java.lang.String value,
java.lang.Long expiration)
Creates a new record in the store with an expiration.
|
protected java.lang.String |
createNamespace(java.lang.String context)
Creates a cache-wide unique namespace for the given context name.
|
boolean |
delete(java.lang.Object value)
Deletes an existing record from the store, using an annotated object as the source.
|
boolean |
delete(java.lang.String context,
java.lang.String key)
Deletes an existing record from the store.
|
void |
deleteContext(java.lang.String context)
Forcibly removes all records in a given context along with any
associated resources devoted to maintaining the context.
|
boolean |
deleteWithVersion(long version,
java.lang.Object value)
Deletes an existing record from the store, using an annotated object as the source, if it
currently has a specified version.
|
boolean |
deleteWithVersion(long version,
java.lang.String context,
java.lang.String key)
Deletes an existing record from the store if it currently has a specified version.
|
protected void |
doDestroy() |
StorageCapabilities |
getCapabilities()
Returns the capabilities of the underlying store.
|
private <T> T |
handleAsyncResult(net.spy.memcached.internal.OperationFuture<T> result) |
protected java.lang.String |
lookupNamespace(java.lang.String context)
Looks up the namespace for the given context name in the cache.
|
private java.lang.String |
memcachedKey(java.lang.String... parts)
Creates a memcached key from one or more parts.
|
java.lang.Object |
read(java.lang.Object value)
Returns an existing record from the store, if one exists, and uses it to
update the annotated fields of a target object.
|
StorageRecord |
read(java.lang.String context,
java.lang.String key)
Returns an existing record from the store, if one exists.
|
net.shibboleth.utilities.java.support.collection.Pair<java.lang.Long,StorageRecord> |
read(java.lang.String context,
java.lang.String key,
long version)
Returns an existing record from the store, along with its version.
|
void |
reap(java.lang.String context)
Manually trigger a cleanup of expired records.
|
void |
setCapabilities(MemcachedStorageCapabilities capabilities)
Sets the storage capabilities.
|
boolean |
update(java.lang.Object value)
Updates an existing record in the store, using an annotated object as the source.
|
boolean |
update(java.lang.String context,
java.lang.String key,
java.lang.Object value,
StorageSerializer serializer,
java.lang.Long expiration)
Updates an existing record in the store using a custom serialization strategy.
|
boolean |
update(java.lang.String context,
java.lang.String key,
java.lang.String value,
java.lang.Long expiration)
Updates an existing record in the store.
|
void |
updateContextExpiration(java.lang.String context,
java.lang.Long expiration)
Updates the expiration time of all records in the context.
|
private boolean |
updateContextKeyList(java.lang.String suffix,
java.lang.String namespace,
java.lang.String key) |
boolean |
updateExpiration(java.lang.Object value)
Updates expiration of an existing record in the store, using an annotated object as the source.
|
boolean |
updateExpiration(java.lang.String context,
java.lang.String key,
java.lang.Long expiration)
Updates expiration of an existing record in the store.
|
java.lang.Long |
updateWithVersion(long version,
java.lang.Object value)
Updates an existing record in the store, if a version matches, using an annotated object as the source.
|
java.lang.Long |
updateWithVersion(long version,
java.lang.String context,
java.lang.String key,
java.lang.Object value,
StorageSerializer serializer,
java.lang.Long expiration)
Updates an existing record in the store, if a version matches, using a custom serialization strategy.
|
java.lang.Long |
updateWithVersion(long version,
java.lang.String context,
java.lang.String key,
java.lang.String value,
java.lang.Long expiration)
Updates an existing record in the store, if a version matches.
|
setIddoInitialize, getIddestroy, initialize, isDestroyed, isInitializedprotected static final java.lang.String CTX_KEY_LIST_SUFFIX
protected static final java.lang.String CTX_KEY_BLACKLIST_SUFFIX
private static final java.lang.String CTX_KEY_LIST_DELIMITER
private static final int MAX_KEY_LENGTH
private final org.slf4j.Logger logger
private final net.spy.memcached.transcoders.Transcoder<MemcachedStorageRecord> storageRecordTranscoder
MemcachedStorageRecord to bytes and vice versa.private final net.spy.memcached.transcoders.Transcoder<java.lang.String> stringTranscoder
@Nonnull private MemcachedStorageCapabilities capabilities
@Nonnull private final net.spy.memcached.MemcachedClient client
@Positive private int timeout
private boolean trackContextKeys
public MemcachedStorageService(@Nonnull
net.spy.memcached.MemcachedClient client,
@Positive
int timeout)
client - Memcached client object. The client MUST be configured to use the binary memcached protocol,
i.e. BinaryConnectionFactory, in order for
deleteWithVersion(long, String, String) and deleteWithVersion(long, Object)
to work correctly. The binary protocol is recommended for efficiency as well.timeout - Memcached operation timeout in seconds.public MemcachedStorageService(@Nonnull
net.spy.memcached.MemcachedClient client,
@Positive
int timeout,
boolean enableContextKeyTracking)
client - Memcached client object. The client MUST be configured to use the binary memcached protocol,
i.e. BinaryConnectionFactory, in order for
deleteWithVersion(long, String, String) and deleteWithVersion(long, Object)
to work correctly. The binary protocol is recommended for efficiency as well.timeout - Memcached operation timeout in seconds.enableContextKeyTracking - True to enable context key tracking, false otherwise. NOTE
this flag must be set to true in order for
updateContextExpiration(String, Long) to work. If that capability is
not needed, the flag should be set to false for better
performance. The feature is disabled by default.@Nonnull public StorageCapabilities getCapabilities()
StorageServicegetCapabilities in interface StorageServicepublic void setCapabilities(@Nonnull
MemcachedStorageCapabilities capabilities)
edu.vt.middleware.idp.storage.MemcachedStorageCapabilities#valueSize should be set equal to the
chosen slab size.capabilities - Memcached storage capabilities.public boolean create(@Nonnull @NotEmpty
java.lang.String context,
@Nonnull @NotEmpty
java.lang.String key,
@Nonnull @NotEmpty
java.lang.String value,
@Nullable @Positive
java.lang.Long expiration)
throws java.io.IOException
StorageServicecreate in interface StorageServicecontext - a storage context labelkey - a key unique to contextvalue - value to storeexpiration - expiration for record, or nulljava.io.IOException - if fatal errors occur in the insertion processpublic boolean create(@Nonnull @NotEmpty
java.lang.String context,
@Nonnull @NotEmpty
java.lang.String key,
@Nonnull
java.lang.Object value,
@Nonnull
StorageSerializer serializer,
@Nullable @Positive
java.lang.Long expiration)
throws java.io.IOException
StorageServicecreate in interface StorageServicecontext - a storage context labelkey - a key unique to contextvalue - object to storeserializer - custom serializer for the objectexpiration - expiration for record, or nulljava.io.IOException - if fatal errors occur in the insertion processpublic boolean create(@Nonnull
java.lang.Object value)
throws java.io.IOException
StorageServiceThe individual parameters for the creation are extracted from the object using the annotations in the org.opensaml.storage.annotation package. If any are missing, or a field inaccessible, a runtime exception of some kind will occur.
create in interface StorageServicevalue - object to storejava.io.IOException - if fatal errors occur in the insertion processpublic StorageRecord read(@Nonnull @NotEmpty java.lang.String context, @Nonnull @NotEmpty java.lang.String key) throws java.io.IOException
StorageServiceread in interface StorageServicecontext - a storage context labelkey - a key unique to contextjava.io.IOException - if errors occur in the read processpublic java.lang.Object read(@Nonnull
java.lang.Object value)
throws java.io.IOException
StorageServiceThe context and key to look up are obtained from the target object, and the value and expiration are written back, using the annotations in the org.opensaml.storage.annotation package. If any are missing, or a field inaccessible, a runtime exception of some kind will occur.
read in interface StorageServicevalue - object to look up and populatejava.io.IOException - if errors occur in the read processpublic net.shibboleth.utilities.java.support.collection.Pair<java.lang.Long,StorageRecord> read(@Nonnull @NotEmpty java.lang.String context, @Nonnull @NotEmpty java.lang.String key, @Positive long version) throws java.io.IOException
StorageServiceThe first member of the pair returned will contain the version of the record in the store, or will be null if no record exists. The second member will contain the record read back. If null, the record either didn't exist (if the first member was also null) or the record was the same version as that supplied by the caller.
read in interface StorageServicecontext - a storage context labelkey - a key unique to contextversion - only return record if newer than supplied versionjava.io.IOException - if errors occur in the read processpublic boolean update(@Nonnull @NotEmpty
java.lang.String context,
@Nonnull @NotEmpty
java.lang.String key,
@Nonnull @NotEmpty
java.lang.String value,
@Nullable @Positive
java.lang.Long expiration)
throws java.io.IOException
StorageServiceupdate in interface StorageServicecontext - a storage context labelkey - a key unique to contextvalue - updated valueexpiration - expiration for record, or nulljava.io.IOException - if errors occur in the update processpublic boolean update(@Nonnull @NotEmpty
java.lang.String context,
@Nonnull @NotEmpty
java.lang.String key,
@Nonnull
java.lang.Object value,
@Nonnull
StorageSerializer serializer,
@Nullable @Positive
java.lang.Long expiration)
throws java.io.IOException
StorageServiceupdate in interface StorageServicecontext - a storage context labelkey - a key unique to contextvalue - updated valueserializer - custom serializerexpiration - expiration for record, or nulljava.io.IOException - if errors occur in the update processpublic boolean update(@Nonnull
java.lang.Object value)
throws java.io.IOException
StorageServiceThe individual parameters for the update are extracted from the object using the annotations in the org.opensaml.storage.annotation package. If any are missing, or a field inaccessible, a runtime exception of some kind will occur.
update in interface StorageServicevalue - object to update fromjava.io.IOException - if errors occur in the update processpublic java.lang.Long updateWithVersion(@Positive
long version,
@Nonnull @NotEmpty
java.lang.String context,
@Nonnull @NotEmpty
java.lang.String key,
@Nonnull @NotEmpty
java.lang.String value,
@Nullable @Positive
java.lang.Long expiration)
throws java.io.IOException,
VersionMismatchException
StorageServiceupdateWithVersion in interface StorageServiceversion - only update if the current version matches this valuecontext - a storage context labelkey - a key unique to contextvalue - updated valueexpiration - expiration for record, or nulljava.io.IOException - if errors occur in the update processVersionMismatchException - if the record has already been updated to a newer versionpublic java.lang.Long updateWithVersion(@Positive
long version,
@Nonnull @NotEmpty
java.lang.String context,
@Nonnull @NotEmpty
java.lang.String key,
@Nonnull
java.lang.Object value,
@Nonnull
StorageSerializer serializer,
@Nullable @Positive
java.lang.Long expiration)
throws java.io.IOException,
VersionMismatchException
StorageServiceupdateWithVersion in interface StorageServiceversion - only update if the current version matches this valuecontext - a storage context labelkey - a key unique to contextvalue - updated valueserializer - custom serializerexpiration - expiration for record, or nulljava.io.IOException - if errors occur in the update processVersionMismatchException - if the record has already been updated to a newer versionpublic java.lang.Long updateWithVersion(@Positive
long version,
@Nonnull
java.lang.Object value)
throws java.io.IOException,
VersionMismatchException
StorageServiceThe individual parameters for the update are extracted from the object using the annotations in the org.opensaml.storage.annotation package. If any are missing, or a field inaccessible, a runtime exception of some kind will occur.
updateWithVersion in interface StorageServiceversion - only update if the current version matches this valuevalue - object to update fromjava.io.IOException - if errors occur in the update processVersionMismatchException - if the record has already been updated to a newer versionpublic boolean updateExpiration(@Nonnull @NotEmpty
java.lang.String context,
@Nonnull @NotEmpty
java.lang.String key,
@Nullable @Positive
java.lang.Long expiration)
throws java.io.IOException
StorageServiceupdateExpiration in interface StorageServicecontext - a storage context labelkey - a key unique to contextexpiration - expiration for record, or nulljava.io.IOException - if errors occur in the update processpublic boolean updateExpiration(@Nonnull
java.lang.Object value)
throws java.io.IOException
StorageServiceThe individual parameters for the update are extracted from the object using the annotations in the org.opensaml.storage.annotation package. If any are missing, or a field inaccessible, a runtime exception of some kind will occur.
updateExpiration in interface StorageServicevalue - object to update fromjava.io.IOException - if errors occur in the update processpublic boolean delete(@Nonnull @NotEmpty
java.lang.String context,
@Nonnull @NotEmpty
java.lang.String key)
throws java.io.IOException
StorageServicedelete in interface StorageServicecontext - a storage context labelkey - a key unique to contextjava.io.IOException - if errors occur in the deletion processpublic boolean delete(@Nonnull
java.lang.Object value)
throws java.io.IOException
StorageServiceThe individual parameters for the deletion are extracted from the object using the annotations in the org.opensaml.storage.annotation package. If any are missing, or a field inaccessible, a runtime exception of some kind will occur.
delete in interface StorageServicevalue - object to deletejava.io.IOException - if errors occur in the deletion processpublic boolean deleteWithVersion(@Positive
long version,
@Nonnull @NotEmpty
java.lang.String context,
@Nonnull @NotEmpty
java.lang.String key)
throws java.io.IOException,
VersionMismatchException
StorageServicedeleteWithVersion in interface StorageServiceversion - record version to deletecontext - a storage context labelkey - a key unique to contextjava.io.IOException - if errors occur in the deletion processVersionMismatchException - if the record has already been updated to a newer versionpublic boolean deleteWithVersion(@Positive
long version,
@Nonnull
java.lang.Object value)
throws java.io.IOException,
VersionMismatchException
StorageServiceThe individual parameters for the deletion are extracted from the object using the annotations in the org.opensaml.storage.annotation package. If any are missing, or a field inaccessible, a runtime exception of some kind will occur.
deleteWithVersion in interface StorageServiceversion - record version to deletevalue - object to deletejava.io.IOException - if errors occur in the deletion processVersionMismatchException - if the record has already been updated to a newer versionpublic void reap(@Nonnull @NotEmpty
java.lang.String context)
throws java.io.IOException
StorageServicereap in interface StorageServicecontext - a storage context labeljava.io.IOException - if errors occur in the cleanup processpublic void updateContextExpiration(@Nonnull @NotEmpty
java.lang.String context,
@Nullable
java.lang.Long expiration)
throws java.io.IOException
StorageServiceupdateContextExpiration in interface StorageServicecontext - a storage context labelexpiration - a new expiration timestamp, or nulljava.io.IOException - if errors occur in the cleanup processpublic void deleteContext(@Nonnull @NotEmpty
java.lang.String context)
throws java.io.IOException
StorageServicedeleteContext in interface StorageServicecontext - a storage context labeljava.io.IOException - if errors occur in the cleanup processprotected void doDestroy()
doDestroy in class net.shibboleth.utilities.java.support.component.AbstractInitializableComponentprotected java.lang.String lookupNamespace(java.lang.String context)
throws java.io.IOException
context - Context name.java.io.IOException - On memcached operation errors.protected java.lang.String createNamespace(java.lang.String context)
throws java.io.IOException
context - Context name.java.io.IOException - On memcached operation errors.private java.lang.String memcachedKey(java.lang.String... parts)
parts - Key parts (i.e. namespace, local name)private <T> T handleAsyncResult(net.spy.memcached.internal.OperationFuture<T> result)
throws java.io.IOException
java.io.IOExceptionprivate boolean updateContextKeyList(java.lang.String suffix,
java.lang.String namespace,
java.lang.String key)
throws java.io.IOException
java.io.IOException