Distributed file systems

''Distributed file systems'' do not share block level access to the same storage but use a network protocol (protocol (computing)). These are commonly known as ''network file systems'', even though they are not the only file systems that use the network to send data. Distributed file systems can restrict access to the file system depending on access lists or capabilities (Capability-based security) on both the servers and the clients, depending on how the protocol is designed.

The difference between a distributed file system and a distributed data store is that a distributed file system allows files to be accessed using the same interfaces and semantics as local files for example, mounting/unmounting, listing directories, read/write at byte boundaries, system's native permission model. Distributed data stores, by contrast, require using a different API or library and have different semantics (most often those of a database).

A distributed file system may also be created by software implementing IBM's Distributed Data Management Architecture (DDM), in which programs running on one computer use local interfaces and semantics to create, manage and access files located on other networked computers. All such client requests are trapped and converted to equivalent messages defined by the DDM. Using protocols also defined by the DDM, these messages are transmitted to the specified remote computer on which a DDM server program interprets the messages and uses the file system interfaces of that computer to locate and interact with the specified file.

# Design goals

Distributed file systems may aim for "transparency" in a number of aspects. That is, they aim to be "invisible" to client programs, which "see" a system which is similar to a local file system. Behind the scenes, the distributed file system handles locating files, transporting data, and potentially providing other features listed below.

* ''Access transparency'' is that clients are unaware that files are distributed and can access them in the same way as local files are accessed. * ''Location transparency''; a consistent name space exists encompassing local as well as remote files. The name of a file does not give its location. * ''Concurrency transparency''; all clients have the same view of the state of the file system. This means that if one process is modifying a file, any other processes on the same system or remote systems that are accessing the files will see the modifications in a coherent manner. * ''Failure transparency''; the client and client programs should operate correctly after a server failure. * ''Heterogeneity''; file service should be provided across different hardware and operating system platforms. * ''Scalability''; the file system should work well in small environments (1 machine, a dozen machines) and also scale gracefully to huge ones (hundreds through tens of thousands of systems). * ''Replication transparency''; to support scalability, we may wish to replicate files across multiple servers. Clients should be unaware of this. * ''Migration transparency''; files should be able to move around without the client's knowledge.

# History

The Incompatible Timesharing System used virtual devices for transparent inter-machine file system access in the 1960s. More file servers were developed in the 1970s. In 1976 Digital Equipment Corporation created the File Access Listener (FAL), an implementation of the Data Access Protocol as part of DECnet Phase II which became the first widely used network file system. In 1985 Sun Microsystems created the file system called "Network File System (Network File System (protocol))" (NFS) which became the first widely used Internet Protocol based network file system.<ref name="ostep-1" /> Other notable network file systems are Andrew File System (AFS), Apple Filing Protocol (AFP), NetWare Core Protocol (NCP), and Server Message Block (SMB) which is also known as Common Internet File System (CIFS).

In 1986, IBM announced client and server support for Distributed Data Management Architecture (DDM) for the System/36, System/38, and IBM mainframe computers running CICS. This was followed by the support for IBM Personal Computer, AS/400, IBM mainframe computers under the MVS and VSE (VSE (operating system)) operating systems, and FlexOS. DDM also became the foundation for Distributed Relational Database Architecture (DRDA), also known as DRDA.

# Examples

* BeeGFS (Fraunhofer) * Ceph (software)|Ceph (Inktank, Red Hat, SUSE) * Distributed File System (Microsoft)|Windows Distributed File System (DFS) (Microsoft) * Infinit (file system)|Infinit * Gfarm file system|GfarmFS * GlusterFS (Red Hat) * Google file system|GFS (Google Inc.) * Hadoop distributed file system|HDFS (Apache Software Foundation) * InterPlanetary File System * iRODS * LizardFS (Skytechnology) * Moose File System|MooseFS (Core Technology / Gemius) * ObjectiveFS * OneFS (EMC Isilon) * OpenIO * OrangeFS (Clemson University, Omnibond Systems), formerly Parallel Virtual File System * Panasas|Panfs (Panasas) * Parallel Virtual File System (Clemson University, Argonne National Laboratory, Ohio Supercomputer Center) * RozoFS (Rozo Systems) * Torus (CoreOS) * XtreemFS