Skwish A blob management library written in Java
Download

Skwish Ranking & Summary

Skwish Tags

manage storage blob manager blob management blobl storage blob

Skwish Description

A blob management library written in Java Skwish is a free and open source Java library that will help you store and retrieve blobs of arbitrary size-entries, in Skwish-speak. The entries (blobs) can contain arbitrary content: to Skwish, every entry is simply an uninterpreted byte sequence. Skwish maintains a simple, fast mapping from numeric entry IDs to entry contents. These entry IDs are determined by Skwish on entry (blob) insertion: the IDs are doled out in ascending order. An application, thus, must maintain the entry IDs somewhere else--typically in an index or a database.So the functionality provided is quite Spartan. This begs the question then, "But what is it good for?" The idea is for Skwish to do one thing, blob storage management, and do it very well. While similar functionality may be found in many existing indexing and database tools (and indeed the file system itself), Skwish is designed to address more niche scenarios.NOTE: Skwish is licensed and distributed under the terms of the Apache License V2.0. Here are some key features of "Skwish": Segment Store: · The library provides an interface to a managed collection of segments. (A segment is the elementary storage unit of the system.) This interface is designed to support multiple concurrent readers together with multiple concurrent writers, and provides all-or-nothing commit semantics. Standalone: · The library also exposes a basic, unmanaged segment implementation. Unmanaged segments are even more lightweight than managed ones. While still thread-safe under concurrent read access, unmanaged segments require a little more care when also writing to them. Committed operations: · Skwish is designed so that when a successfully completed write operation returns (i.e. one that doesn't raise an exception), the changes are guaranteed to be written out to persistent storage (to the extent the operating system and other things like the device controller allow). This is true whether or not the write occurs in the context of a transaction (though transactions do provide much stronger guarantees.) Abnormal shutdown: · The system is relatively fail-safe in the face of a crash or other abnormal shutdown. That is, if a running instance is abruptly terminated, chances are very good that the system will still be in a consistent state on restart. We say, "chances are very good," because there is still a small window (the partial write of an 8 byte value denoting the entry count in a segment's index file) in which abrupt termination can result in data corruption. (This hole will be plugged.) java.nio: · Skwish leverages the lower level I/O abstractions Java exposes under the java.nio packages. These abstractions (e.g. FileChannels and ByteBuffer) are not just used under the hood; they are exposed all the way out in Skwish's public API. The goal is to allow layering efficient applications on top Skwish. The library's new experimental non-blocking HTTP interface is an example of such an application. Access methods: · Skwish provides 2 basic ways for reading and writing entry contents. One is value-based and involves copying entry contents to and from memory (via a ByteBuffer, see below). This method is typically suitable for accessing smaller entries. The second representation of entry contents is stream-based, and is especially suitable for accessing larger entries: obtaining a reference to an entry-stream (a FileChannel) costs at most one disk seek, and depending on how the stream is used, little to none of the actual entry contents need ever be loaded (copied) into memory (as when, for example, the contents is to be piped to another channel). Requirements: · JRE 1.5 or later · JDK1.5 or later What's New in This Release: · Added TxnSegment.getEntryInsertionChannel() method. · Added better setup for the experimental web server Caroon. · Refactored the directory structure traverser in order to abstract away the tree structure. · More documentation and a follow up article in the "getting started" guide.

Skwish Related Software