Skip navigation.
Home
PersistentFS.com

PersistentFS v0.9 Release Notes

PersistentFS v0.9 Technology Preview

Release Notes

Latest Release: 11 August 2008

PersistentFS is a POSIX-compliant file system that provides essentially unlimited storage backed by Amazon Simple Storage Service (Amazon S3).

This is a technology preview release intended to allow potential users to assess the technology and its usability in their application. Although extensive synthetic tests have been run, the file system has not been subject to sufficient live testing to have confidence there are no serious bugs. PersistentFS.com intends to run additional tests, especially multi-threaded tests, to ensure the file system will operate reliably under load.

The current release is free to use. It is our goal to deliver a commercial version in the near future with anticipated pricing of approximately $20 per month for 1 TB of storage. Until the commercial version is released, developers are welcome to use our free releases. The latest release expires on 2008-Oct-15. We will continue extending the free releases until the commercial release is available. To receive announcements of new releases, please sign up for our email list.

This release of PersistentFS has a number of limitations:

  • The file system currently supports symbolic links, but not hard links. (Hard links are optional in the POSIX standard.) We plan to implement hard links in a future release.
  • This version of PersistentFS does not implement the standard file timestamp values st_atime, st_mtime and st_ctime. When a file or directory is created, these values are set to the creation time and then never updated by PersistentFS. They may however be updated by the operating system and other utilities.
  • PersistentFS does not include entries for "." and ".." in its index. According to the POSIX standard, these entries are optional, and when they are not included in the index, then the file system must not include them in a directory listing. PersistentFS complies with the POSIX standard in this respect, and does not return entries for "." and "..". Many utilities automatically generate equivalent pseudo-entries.
  • PersistentFS internally uses 64-bit inode numbers. At present, FUSE creates temporary 32-bit inode number for all files. Eventually, this may be changed.
  • When performing a "stat" on a file, for compatibility purposes PersistentFS always reports a block size of 512 and computes the number of blocks based on the file size divided by 512. The size of a directory is always reported as 30 bytes. The number of links for all file types is always reported as 1.
  • PersistentFS has an essentially unlimited capacity and therefore has no concept of "blocks available", nor does it track the number of "blocks" in use. For compatibility purposes, PersistentFS always reports that the file system contains 2^48 bytes, all of which are available.
  • This version of PersistentFS does not support sparse files. If you leave a gap when writing a file, it is automatically filled with zeros (as required by the POSIX standard), and these zeros are written to S3 in the same way as other file data blocks.
  • To mitigate against S3's "eventual consistency" problem, the file system does not currently discard any file index pages from virtual memory. If the file system is very large, this may eventually lead to the exhaustion of virtual memory. The amount of memory used is approximately 50 bytes per inode.
  • This version of PersistentFS contains numerous "assertion" checks. If one of these assertions fails, a message is logged and the operation will return a -999 error. In some cases, conditions that may occur in normal operation will return -999 errors when they should return more descriptive error codes. In addition, if an assertion failure occurs while modifying file system data structures or while writing modified data to S3, the file system will "panic" and enter read-only mode. This read-only mode is intended to protect the data already in the file system from corruption. If you receive a -999 error or a file system panic, we kindly request that you report this to our Technical Support department.
  • PersistentFS uses the FUSE high-level interface. In some situations, the FUSE high-level interface may block read operations while a write operation is in progress, even though the reads are not being blocked by PersistentFS. For higher performance and greater parallelism, we plan to investigate migrating to the FUSE low-level interface in the future.

Known Issues

The following is a list of known issues in the most recent revision of PersistentFS, which identifies itself as "Built Aug 11 2008 10:41:24". The build time can be determined by running PersistentFS and looking at the 8th line of its output.

  • In the current Technology Preview release, changes to an open file's meta data are not saved to S3 until the file is closed. As a result, if PersistentFS or the system crashes while writing a file, it is possible for the file size in the file's directory entry to be greater than the actual number of file blocks written to S3. If this happens, the next time the file system is mounted and an attempt is made to access a file block that is not stored on S3, the file system will panic and become read-only. This will be fixed in a future release. In the meantime, one work around is to delete the file or truncate it to reset its size.