File System Development

1. Indroduction
2. Interface
3. File System Registration
4. Configuration Script
5. Handling Devices and Pipes
6. Summary

1. Introduction

The file systems are used to store regular, device-like, and FIFO files. Some file systems do not support special files, e.g. pipes and devices, because is not possible to handle this kind of files (e.g. on the FAT) in the particular file system. In the system there are few file systems:

  • lfs – the file system is a general usage file system that store files in the RAM. This file system is used by the system by default,
  • devfs – the file system is used only to handle devices, it is much faster than lfs, but does not provide many features,
  • procfs – the file system is a special file system that provides the system information in form of the files,
  • fatfs – the file system is used to read all storages that contains FAT12, FAT16, and FAT32 file systems.

As we can see, each file system has special role in the system. In this case, manage of the file systems must be the same for file that can comes from different sources. To achieve this requirement the system has implemented core component called VFS. The VFS is a kind of router that translates the user’s path to the specified file system. The usage of files for the user is transparent; the user do not care about real file position.

As mentioned above, each file system must be connected to the VFS. To do this, the file systems must have the same interfaces. Thus, any file system can be mounted easily by the VFS. File systems are similar to driver modules, because single file system can handle many source files that contains specified file system format (e.g. FAT, ext2, etc).

2. Interface

The file systems are handled by specified interface functions:

  • API_FS_INIT() – the function initializes the file system.
  • API_FS_RELEASE() – the function releases allocated/used resource of the file system.
  • API_FS_MKNOD() – the function creates a device node.
  • API_FS_MKDIR() – the function creates a folder.
  • API_FS_MKFIFO() – the function creates a FIFO file (pipe).
  • API_FS_OPENDIR() – the function opens a specified directory.
  • API_FS_REMOVE() – the function removes a specified file (regular file, folder, pipe, and device).
  • API_FS_RENAME() – the function renames a specified file.
  • API_FS_CHMOD() – the function changes a mode of specified file.
  • API_FS_CHOWN() – the function changes an owner of specified file.
  • API_FS_STAT() – the function gets a specified file statistics (by a path of a file).
  • API_FS_FSTAT() – the function gets a specified file statistics (by a file descriptor).
  • API_FS_STATFS() – the function gets the file system statistics.
  • API_FS_OPEN() – the function opens a specified file by using its path.
  • API_FS_CLOSE() – the function closes a specified file.
  • API_FS_WRITE() – the function writes a data to a specified file.
  • API_FS_READ() – the function reads a data from a specified file.
  • API_FS_IOCTL() – the function performs an operations on a file, that are not precised by the interface.
  • API_FS_FLUSH() – the function forces a buffer write of a file.
  • API_FS_SYNC() – the function synchronizes the file system with its medium.

The interface functions are created by macros, thanks this the name of file system functions are hidden and known only by the system. The return type of the functions are also hidden, but to the better understanding of code the returned type is enclosed in square brackets.

2.1. The API_FS_INIT()

[std_ret_t] API_FS_INIT(fs_name,
                        void **fs_handle,
                        const char *src_path)

The function is called when the operating system, on the user request (by the mount() function), mounts a file system from specified file. Arguments:

fs_name a name of the file system
fs_handle a pointer to the memory region allocated by the file system. This is the output pointer. A data from this pointer is used to pass the file system instance;
src_path a data source file (a medium source).

When the file system was correctly initialized then the STD_RET_OK is returned, otherwise the STD_RET_ERROR and appropriate errno value will be set.

2.2. The API_FS_RELEASE()

[std_ret_t] API_FS_RELEASE(fs_name,
                           void *fs_handle)

The function is called when the operating system, on the user request (by the umount() function), unmounts a file system. Arguments:

fs_name a name of the file system
fs_handle a pointer to the memory region allocated by the file system (the file system instance).

When the file system was correctly released then the STD_RET_OK is returned, otherwise the STD_RET_ERROR and appropriate errno value will be set.

2.3. The API_FS_MKNOD()

[std_ret_t] API_FS_MKNOD(fs_name,
                         void *fs_handle,
                         const char *path,
                         const dev_t dev)

The function is called when the operating system, on the user request (by the mknod() function), creates the node for a device. Arguments:

fs_name a name of the file system
fs_handle a pointer to the memory region allocated by the file system (the file system instance);
path a path where a node will be created;
dev a device number.

When the node was correctly created then the STD_RET_OK is returned, otherwise the STD_RET_ERROR and appropriate errno value will be set.

2.4. The API_FS_MKDIR()

[std_ret_t] API_FS_MKDIR(fs_name,
                         void *fs_handle,
                         const char *path,
                         mode_t mode)

The function was called when the operating system, on the user request (by the mkdir() function), creates a new folder. Arguments:

fs_name a name of the file system
fs_handle a pointer to the memory region allocated by the file system (the file system instance);
path a path where a node will be created;
mode a mode of the created folder.

When the folder was correctly created then the STD_RET_OK is returned, otherwise the STD_RET_ERROR and appropriate errno value will be set.

2.5. The API_FS_MKFIFO()

[std_ret_t] API_FS_MKFIFO(fs_name,
                          void *fs_handle,
                          const char *path,
                          mode_t mode)

The function is called when the operating system, on the user request (by the mkfifo() function), creates a new FIFO file (pipe). Arguments:

fs_name a name of the file system
fs_handle a pointer to the memory region allocated by the file system (the file system instance);
path a path where a node will be created;
mode a mode of the created FIFO.

When the FIFO was correctly created then the STD_RET_OK is returned, otherwise the STD_RET_ERROR and appropriate errno value will be set.

2.6. The API_FS_OPENDIR()

[std_ret_t] API_FS_OPENDIR(fs_name,
                           void *fs_handle,
                           const char *path,
                           DIR *dir)

The function is called when the operating system, on the user request (by the opendir() function), opens a directory. Arguments:

fs_name a name of the file system
fs_handle a pointer to the memory region allocated by the file system (the file system instance);
path a path where a node will be created;
dir an output pointer that sets directory object.

When the directory was correctly opened then the STD_RET_OK is returned, otherwise the STD_RET_ERROR and appropriate errno value will be set.

2.7. The API_FS_REMOVE()

[std_ret_t] API_FS_REMOVE(fs_name,
                          void *fs_handle,
                          const char *path)

The function is called when the operating system, on the user request (by the remove() function), removes a file. Arguments:

fs_name a name of the file system
fs_handle a pointer to the memory region allocated by the file system (the file system instance);
path a path to the removed file.

When the file was correctly removed then the STD_RET_OK is returned, otherwise the STD_RET_ERROR and appropriate errno value will be set.

2.8. The API_FS_RENAME()

[std_ret_t] API_FS_RENAME(fs_name,
                          void *fs_handle,
                          const char *old_name,
                          const char *new_name)

The function is called when the operating system, on the user request (by the rename() function), renames a file. Arguments:

fs_name a name of the file system
fs_handle a pointer to the memory region allocated by the file system (the file system instance);
old_name an old file name (path);
new_name a new file name (path).

When the file was correctly renamed then the STD_RET_OK is returned, otherwise the STD_RET_ERROR and appropriate errno value will be set.

2.9. The API_FS_CHMOD()

[std_ret_t] API_FS_CHMOD(fs_name,
                         void *fs_handle,
                         const char *path,
                         mode_t mode)

The function is called when the operating system, on the user request (by the chmod() function), changes a mode of a file. Arguments:

fs_name a name of the file system
fs_handle a pointer to the memory region allocated by the file system (the file system instance);
path a path to a file;
mode a new mode.

When the mode was correctly changed then the STD_RET_OK is returned, otherwise the STD_RET_ERROR and appropriate errno value will be set.

2.10. The API_FS_CHOWN()

[std_ret_t] API_FS_CHOWN(fs_name,
                         void *fs_handle,
                         const char *path,
                         uid_t uid,
                         gid_t gid)

The function is called when the operating system, on the user request (by the chown() function), changes a user and a group of a file. Arguments:

fs_name a name of the file system
fs_handle a pointer to the memory region allocated by the file system (the file system instance);
path a path to a file;
uid a new user ID,
gid a new group ID.

When the owner was correctly changed then the STD_RET_OK is returned, otherwise the STD_RET_ERROR and appropriate errno value will be set.

2.11. The API_FS_STAT()

[std_ret_t] API_FS_STAT(fs_name,
                        void *fs_handle,
                        const char *path,
                        struct stat *stat)

The function is called when the operating system, on the user request (by the stat() function), gets an information of a file. Arguments:

fs_name a name of the file system
fs_handle a pointer to the memory region allocated by the file system (the file system instance);
path a path to a file;
stat a pointer to the structure where an information must be stored.

When the information was correctly updated then the STD_RET_OK is returned, otherwise the STD_RET_ERROR and appropriate errno value will be set.

2.12. The API_FS_FSTAT()

[std_ret_t] API_FS_FSTAT(fs_name,
                         void *fs_handle,
                         void *extra,
                         fd_t fd,
                         struct stat *stat)

The function is called when the operating system, on the user request (by the fstat() function), gets an information of a file. Arguments:

fs_name a name of the file system
fs_handle a pointer to the memory region allocated by the file system (the file system instance);
extra an extra data paired with a opened file descriptor;
fd a file descriptor understood by the file system;
stat a pointer to the structure where an information must be stored.

When the information was correctly updated then the STD_RET_OK is returned, otherwise the STD_RET_ERROR and appropriate errno value will be set.

2.13. The API_FS_STATFS()

[std_ret_t] API_FS_STATFS(fs_name,
                          void *fs_handle,
                          struct statfs *statfs)

The function is called when the operating system, on the user request (by the statfs() function), gets an information of the file system. Arguments:

fs_name a name of the file system
fs_handle a pointer to the memory region allocated by the file system (the file system instance);
statfs a pointer to the structure where an information must be stored.

When the information was correctly updated then the STD_RET_OK is returned, otherwise the STD_RET_ERROR and appropriate errno value will be set.

2.14. The API_FS_OPEN()

[std_ret_t] API_FS_OPEN(fs_name,
                        void *fs_handle,
                        void **extra,
                        fd_t *fd,
                        fpos_t *fpos,
                        const char *path,
                        vfs_open_flags_t flags)

The function is called when the operating system, on the user request (by the fopen() function), opens a file. Arguments:

fs_name a name of the file system
fs_handle a pointer to the memory region allocated by the file system (the file system instance);
extra an output pointer to an extra data coupled with a opened file;
fd an output pointer to a file descriptor understood by the file system;
fpos an output pointer to a file position;
path a path of a file to open;
flags a flags that indicate a file open operations.

When the file was correctly opened then the STD_RET_OK is returned, otherwise the STD_RET_ERROR and appropriate errno value will be set.

2.15. The API_FS_CLOSE()

[std_ret_t] API_FS_CLOSE(fs_name,
                         void *fs_handle,
                         void *extra,
                         fd_t fd,
                         bool force)

The function is called when the operating system, on the user request (by the fclose() function), closes a opened file. Arguments:

fs_name a name of the file system
fs_handle a pointer to the memory region allocated by the file system (the file system instance);
extra an extra data coupled with a file descriptor;
fd a file descriptor understood by the file system;
force a flag that indicates the force-close of the file.

When the file was correctly closed then the STD_RET_OK is returned, otherwise the STD_RET_ERROR and appropriate errno value will be set.

2.16. The API_FS_WRITE()

[ssize_t] API_FS_WRITE(fs_name,
                       void *fs_handle,
                       void *extra,
                       fd_t fd,
                       u8_t *src,
                       size_t count,
                       fpos_t *fpos,
                       struct vfs_fattr fattr)

The function is called when the operating system, on the user request (by the fwrite() function), writes a data to a file. Arguments:

fs_name a name of the file system
fs_handle a pointer to the memory region allocated by the file system (the file system instance);
extra an extra data coupled with a file descriptor;
fd a file descriptor understood by the file system;
src a data source pointer;
count a number of bytes to write;
fpos a file position;
fattr a write operation attributes.

On success, returns a number of written bytes. On error, -1 is returned and appropriate errno value will be set.

2.17. The API_FS_READ()

[ssize_t] API_FS_READ(fs_name,
                       void *fs_handle,
                       void *extra,
                       fd_t fd,
                       u8_t *src,
                       size_t count,
                       fpos_t *fpos,
                       struct vfs_fattr fattr)

The function is called when the operating system, on the user request (by the fread() function), reads a data from a file. Arguments:

fs_name a name of the file system
fs_handle a pointer to the memory region allocated by the file system (the file system instance);
extra an extra data coupled with a file descriptor;
fd a file descriptor understood by the file system;
dst a data destination pointer;
count a number of bytes to read;
fpos a file position;
fattr a write operation attributes.

On success, returns a number of read bytes. On error, -1 is returned and appropriate errno value will be set.

2.18. The API_FS_IOCTL()

[stdret_t] API_FS_IOCTL(fs_name,
                       void *fs_handle,
                       void *extra,
                       fd_t fd,
                       int request,
                       void *arg)

The function is called when the operating system, on the user request (by the ioctl() function), performs a special operation on a file. Arguments:

fs_name a name of the file system
fs_handle a pointer to the memory region allocated by the file system (the file system instance);
extra an extra data coupled with a file descriptor;
fd a file descriptor understood by the file system;
request an operation request;
arg an argument of a request.

The return behavior is depending on the file type (devce or pipe). A some ioctls returns the STD_RET_OK or the STD_RET_ERROR, but can also returns a negative values for errors and a positive value for the valid operations.

2.19. The API_FS_FLUSH()

[stdret_t] API_FS_FLUSH(fs_name,
                       void *fs_handle,
                       void *extra,
                       fd_t fd)

The function is called when the operating system, on the user request (by the fflush() function), forces a buffer write to a medium. Arguments:

fs_name a name of the file system
fs_handle a pointer to the memory region allocated by the file system (the file system instance);
extra an extra data coupled with a file descriptor;
fd a file descriptor understood by the file system.

2.20. The API_FS_SYNC()

[stdret_t] API_FS_SYNC(fs_name,
                       void *fs_handle)

The function is called when the operating system, on the user request (by the sync() function), forces the buffers write of all opened files to a medium. Arguments:

fs_name a name of the file system
fs_handle a pointer to the memory region allocated by the file system (the file system instance).

3. File System Registration

When the file system was created, then must be registered in the system register. To register the file system, follow:

  1. Go to the ./src/system/fs directory, and open your new file system folder. Create the Makefile file, and add specified content:
    # Makefile for GNU make
    CSRC_CORE += fs/your_fs/your_fs.c
        

    The content adds specified files of the file system to the build process,

  2. Close this file, and open the ./src/system/fs/Makefile file, and add specified lines:
    ifeq ($(ENABLE_YOUR_FS), __YES__)
    include $(FS_LOC)/your_fs/Makefile
    endif
        
  3. Close this file. Open the ./src/system/fs/fs_registration.c file, and in the External objects section add specified content:
    #if (__ENABLE_YOUR_FS__)
    _IMPORT_FILE_SYSTEM(your_fs);
    #endif
        
  4. Next, find the _FS_table[] array, and add specified line:
    #if (__ENABLE_YOUR_FS__)
    _FILE_SYSTEM_INTERFACE(your_fs),
    #endif
        

    save this file and close.

From this time, your file system will be visible in the system, and can be used to mount specified medium.

4. Configuration Script

The file system will be compiled only when the __ENABLE_YOUR_FS__ flags is set to __YES__. In this purpose, follow:

  1. Open the ./config/project/flags.h file, and find the file systems section, and add specified definition:
    #define __ENABLE_YOUR_FS__ __YES__
        

    This flags will be modified by configuration script,

  2. Save and close this file and open the ./config/project/Makefile file, and in the file system enable flags section add specified line:
    ENABLE_YOUR_FS=__YES__
        

    From this time your file system will be compiled, because is enabled.

  3. Save and close this file and open the ./tools/wizard/modules/fs.lua file, and add specified function:
    local function configure_your_fs_enable()
            local choice = key_read(db.path.project.flags, "__ENABLE_YOUR_FS__")
            msg(progress() .. "Put here your file system description.")
            msg("Do you want to enable your fs?")
            msg("Current choice is: " .. filter_yes_no(choice) .. ".")
            add_item(yes, "YES")
            add_item(no, "NO")
            choice = get_selection()
            if (can_be_saved(choice)) then
                    key_save(db.path.project.flags, "__ENABLE_YOUR_FS__", choice)
                    key_save(db.path.project.mk,    "ENABLE_YOUR_FS",     choice)
            end
    
            return choice
    end
        

    This function creates wizard page that will be shown in the configuration.

  4. Find the fs:configure function and add to the pages variable your function, as follow:
    -- ...
    
    local pages = {configure_devfs_enable,
                   configure_lfs_enable,
                   configure_fatfs_enable,
                   configure_procfs_enable,
                   configure_your_fs_enable, -- add this line
                   print_summary}
    
    -- ...
        

From this time, your file system can be enabled or disabled from configuration wizard.

5. Handling Devices and Pipes

The dnx RTOS provide several functions that must be used to handle devices and pipes. To handle devices use the core/fs.h header and following functions:

  • driver_open() – the function opens a selected device;
  • driver_close() – the function close a selected device;
  • driver_write() – the function write data to a selected device;
  • driver_read() – the function read data from a selected device;
  • driver_ioctl() – the function does a device’s non-standard operation;
  • driver_flush() – the function flushes a device’s buffer;
  • driver_stat() – the function gets a device information.

The functions are similar to methods that are implemented in the each module. Please refer Driver development chapter to obtain how to use those functions. To learn more, examine the source code of lfs and devfs file systems. The drivers in the file systems are created by using mknod() function.

To use pipes in the file system, use specified functions from core/pipe.h header:

  • pipe_new – the function creates new pipe object;
  • pipe_delete – the function destroy selected pipe object;
  • pipe_get_length – the function gets pipe’s length (capacity);
  • pipe_read – the function reads a data from the pipe;
  • pipe_write – the function writes a data to the pipe;
  • pipe_close – the function closes the pipe. If a pipe is closed, then no more bytes can be read from, and program detect this file as empty. The closed pipe can not be opened again.

The pipes in the file systems are created by using mkfifo() function (the file system method). To obtain usage of pipe functions, examine the lfs and devfs file systems.

6. Summary

The implementation of file systems are more difficult than e.g. drivers, because the file systems are more complex. The best way to understood how to create own file system is a practice. The presented functions/methods are similar in the each file system, so implementation of a new file system may not be difficult, because in the Internet there are many ready implementations. To learn more examine the ./src/system/fs folder.