iRODS A new data grid software system.
Download

iRODS Ranking & Summary

iRODS Tags

grid Data Grid Grid Security Infrastructure SRB technology GSI SRB

iRODS Description

A new data grid software system. iRODS is a new data grid application system developed by the San Diego Supercomputer Center (SDSC) Storage Resource Broker (SRB) team and collaborators. The system is based upon the expertise gained through the application of the SRB technology in support of data grids, digital libraries, persistent archives, and real-time data systems.The management policies (sets of assertions that these communities make about their collections) are being characterized in terms of rules and state information. At the iRODS core, a Rule Engine interprets rules to decide how the system is to respond to various requests and conditions. iRODS is open source (under a BSD-type license).We are supporting SDSC SRB while developing iRODS in parallel, and will do so for quite some time to come. At some point, we will provide tools to migrate from SDSC SRB to iRODS for those who wish to do so. What's New in This Release: · Federation. Zone Federation has been added which provides facilities for two or more independent iRods systems to interact with each other and allow for seamless access of data and metadata across these iRods systems. These systems are called iRODS Zones, with each Zone running its own iCat and administrative domain. iRODS Federation requires less synchronization between zones than the similar SRB Federation system. See Federation and Federation Administration for more information. · Master/Slave iCat with Oracle. An iRods installation or Zone can be configured to run with a single Master iCat plus zero or more Slave iCats. The purpose of the Slave iCat is to improve responsiveness of queries across a Wide-Area-Network. The Slave iCats are used for "read only" type queries. The following icommands have been converted to use the Slave iCat by default: icd, iget, ils, ilsresc and iqstat. This makes use of RDBMS functionality to sychronize the ICAT databases and so is available when using Oracle as the ICAT RDBMS. · Initial SRB to iRODS Migration tool. There is now a preliminary version of a tool to help convert an SRB Instance to an iRods one. The 'm2icat.pl' script uses Spullmeta to get SRB-MCAT information and creates and executes sets of commands for iadmin, psql, and imeta to create resources, collections, dataObjects, and users in the iRods instance. The iRODS system can then access former-SRB data without moving the physical files. This is still incomplete and cannot handle many of the features of SRB, but you may wish to experiment with it. See the script for more information and contact us to help us plan additional extensions. · Grid Security Infrastructure (GSI) Improvements. A significant problem in the iRODS interface to GSI was corrected, allowing regular iRODS users to authenctiate with GSI. Users can also now set the environment variable SERVER_DN to authenticate the server via the GSI system (perform mutual authentication). · iRods FUSE improvements. iRods FUSE now works with the latest versions of FUSE instead of only version 2.7.0. Caching files and directories query results to improve the performance of the getattri call which is call frequently by FUSE. Small files are cached to improve the I/O performance on small files. File modes of files has been enabled so that chmod of files now works. You should see noticeable performance improvement in commands such as ls or cp of small files. · iRODS Explorer for Windows. The new iRODS Explorer for Windows has been available since mid-September. As described on the windows page, this is an iRODS browser that runs as a native Windows binary and provides a rich Graphical User Interface and a fast navigation of the hierarchical collection-file structure inside iRODS. In addition, users can add, modify, and view metadata with long string values through a user-friendly metadata dialog. · DataMode preserved. A "dataMode" metadata item was added so that the file mode of files uploaded to iRods and downloaded from iRods, can be preserved. "dataMode" is the Read-Write-Execute status for user, group, other, of a file. · New bundling. A new 'ibun' command is used to handle the bundling of small files into structured files such as tar files. It can be used for the uploading, downloading, and archival of a large number of small files. For example, to upload a large number of small files, a user can use the normal UNIX tar command to tar these files into a single tar file. This single tar file can then be uploaded to iRods using the iput command. The "ibun -x" command can be used to request the iRods server to untar this file into many small files and register these small files with the iCat. Similarly the "ibun -c" command can be used to efficiently download a large number of small files. · New 'ipasswd' Command. 'ipasswd' allows users to change their iRODS password. As with the corresponding iadmin command, the password is obfuscated for network transfer. · Rule-oriented Data Access (RDA) ported to Oracle. RDA is now supported on Oracle as well as the previous PostgreSQL. RDA provides access to arbitrary databases through the iRODS system, somewhat like the SRB DAI (Database Access Interface) but implemented via rules and micro-services. · Other RDA improvements. A msiRdaRollback micro-service was implemented; some memory leaks were fixed; and an obfuscated form of the RDA password can be set in the RDA configuration file. · Rule-language 'break'. A 'break' statement is now accepted in the iRODS Rule language to break out of for, while, and foreach loops. · Federation User and Administrative Changes. In the iadmin and imeta commands, users are now represented with an optional Zone name (user), where the local Zone is default. iadmin has mkzone, modzone, and rmzone subcommands to manipulate remote-zone information. ilsresc, imeta, and irmtrash now have '-z zoneName' options to work with remote Zones. Internally, user authentication uses an optional Zone name. · Federation Server Authentication. A capability was added where the iRODS Server which is authenticating a Client for a remote Zone, is itself authenticated. This is optional, but highly recommended for Federated Zones to improve security. See -- for additional information. · Zone Renaming. The iadmin tool can now be used to rename your local Zone, handling the conversion of the Zone and user information and renaming user home collections (via a new Rule and micro-services). This may be useful with iRODS Zone Federation. · Timed connection -T option. A new -T option was added to the iput, iget, irepl and icp commands which renews the socket connection between the client and server after 10 minutes of connection. This gets around the problem of sockets getting timed out by the firewall as reported by some users. · New RBUDP data transfer mode. A new data transfer mode - RBUDP (Reliable Blast UDP) was added, in addition to the existing the sequential (single TCP stream) and parallel (multi TCP streams) modes currently supported by iRODS. RBUDP is developed by Eric He, Jason Leigh, Oliver Yu and Thomas Defanti of U of Ill at Chicago. http://www.evl.uic.edu/cavern/RBUDP/Reliable Blast UDP.html It uses the UDP protocol for high performance data transfer. A new option -Q has been added to the iput, iget,irepl and icp command to specify the use of the RBUDP protocol. · HDF5/iRods Improvements. The HDF5/iRods client can now be built without linking to the HDF5 library. A JNI capability for JAVA client such a hdfView, was added. A memory leak was fixed. · Inherited Access Permissions. An inheritance attribute can be set on a collection to cause new data-Objects and sub-collections created under it to acquire the access rights (ACLs) of the collection. See 'ichmod -h' and 'ils -A' for more information. · ICAT Improvements. The iRODS Metadata Catalog interface software (ICAT) was improved in various ways, in addition to changes for iRODS Zone Federation. Access to the Audit tables via queries is now restricted by default. A bug was fixed dealing with recursively setting access control on replicated data-objects. Problems in getting the totalRowCount (when requested) were resolved (for both Oracle and Postgres). In the General-Query, any number of compound conditions (separated by || or &&) can now be handled (instead of just two). After various errors, the ICAT functions (when using PostgreSQL) will do an automatic rollback to allow subsequent SQL to function. Some ICAT-Oracle memory leaks were found and fixed. imeta and the ICAT AVU queries can now accept multiple conditions separated by 'and' and a single 'or'. 'isysmeta' can now set the data-type of a data-object. For improved long-term maintenance, internal changes were made in the way that the ICAT general queries are structured. · Additional Micro-services. A number of new micro-services have been added to enable new functionality, including some of the features described in these release notes. These can also be used in your own rules and as examples for developing your own micro-services. See Released Micro Services for the current list. Special thanks to Romain Guinot of the Open Source Center - Atos Origin (http://www.portaildulibre.fr) for providing some of the new micro-services, both for the core and as a separate module ('guinot'). · Testing Improvements. As before, the IRODS development version is continually built and tested on local DICE hosts and occasionally on various platforms at the NMI Build & Test facility. ICAT tests were expanded to cover new ICAT functionality. Some heavy-load tests were developed. · Installation/Control Improvements. A variety of changes were made to the iRODS installation and control scripts to handle specific error situations; also, finishSetup.pl will now update ~/.odbc.ini to include the section that is also stored in the .../pgsql/etc/odbc.ini file, to avoid problems on some hosts, etc. Also, the Make scripts will now automatically re-link modules that have updated source files. And 'iinit' will create the ~/.irods directory if it is not present and prompt for and store the needed .irodsEnv items if they are missing. The install scripts now default to Postgres 8.3.5. · New Transfer Logging. When enabled (manually), the transfer operations of get, put, replicate, and rsync are logged. See the comments in rsDataObjClose.c for more information. · imkdir can create parent collection. A -p option was added to the imkdir command to cause it to create parent collections if they don't already exist. · Other Bug Fixes. As always, other bugs have been fixed and many small improvements made; too numerous to describe.

iRODS Related Software