jimzah/freebsd-src
1
0
Fork 0
mirror of https://github.com/freebsd/freebsd-src.git synced 2024-11-30 08:43:23 +00:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
8d457988a7
freebsd-src/sbin/newfs/newfs.h

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

126 lines
5.5 KiB
C
Raw Normal View History

General further adoption of SPDX licensing ID tags. Mainly focus on files that use BSD 3-Clause license. The Software Package Data Exchange (SPDX) group provides a specification to make it easier for automated tools to detect and summarize well known opensource licenses. We are gradually adopting the specification, noting that the tags are considered only advisory and do not, in any way, superceed or replace the license texts. Special thanks to Wind River for providing access to "The Duke of Highlander" tool: an older (2014) run over FreeBSD tree was useful as a starting point.
2017-11-20 19:49:47 +00:00
/*-
* SPDX-License-Identifier: BSD-3-Clause
*
This commit adds basic support for the UFS2 filesystem. The UFS2 filesystem expands the inode to 256 bytes to make space for 64-bit block pointers. It also adds a file-creation time field, an ability to use jumbo blocks per inode to allow extent like pointer density, and space for extended attributes (up to twice the filesystem block size worth of attributes, e.g., on a 16K filesystem, there is space for 32K of attributes). UFS2 fully supports and runs existing UFS1 filesystems. New filesystems built using newfs can be built in either UFS1 or UFS2 format using the -O option. In this commit UFS1 is the default format, so if you want to build UFS2 format filesystems, you must specify -O 2. This default will be changed to UFS2 when UFS2 proves itself to be stable. In this commit the boot code for reading UFS2 filesystems is not compiled (see /sys/boot/common/ufsread.c) as there is insufficient space in the boot block. Once the size of the boot block is increased, this code can be defined. Things to note: the definition of SBSIZE has changed to SBLOCKSIZE. The header file <ufs/ufs/dinode.h> must be included before <ufs/ffs/fs.h> so as to get the definitions of ufs2_daddr_t and ufs_lbn_t. Still TODO: Verify that the first level bootstraps work for all the architectures. Convert the utility ffsinfo to understand UFS2 and test growfs. Add support for the extended attribute storage. Update soft updates to ensure integrity of extended attribute storage. Switch the current extended attribute interfaces to use the extended attribute storage. Add the extent like functionality (framework is there, but is currently never used). Sponsored by: DARPA & NAI Labs. Reviewed by: Poul-Henning Kamp <phk@freebsd.org>
2002-06-21 07:18:05 +01:00
* Copyright (c) 2002 Networks Associates Technology, Inc.
* All rights reserved.
*
* This software was developed for the FreeBSD Project by Marshall
* Kirk McKusick and Network Associates Laboratories, the Security
* Research Division of Network Associates, Inc. under DARPA/SPAWAR
* contract N66001-01-C-8035 ("CBOSS"), as part of the DARPA CHATS
Correct lines incorrectly added to the copyright message. Add missing period. Submitted by: Bruce Evans <bde@zeta.org.au> Sponsored by: DARPA & NAI Labs.
2003-02-14 21:08:14 +00:00
* research program.
This commit adds basic support for the UFS2 filesystem. The UFS2 filesystem expands the inode to 256 bytes to make space for 64-bit block pointers. It also adds a file-creation time field, an ability to use jumbo blocks per inode to allow extent like pointer density, and space for extended attributes (up to twice the filesystem block size worth of attributes, e.g., on a 16K filesystem, there is space for 32K of attributes). UFS2 fully supports and runs existing UFS1 filesystems. New filesystems built using newfs can be built in either UFS1 or UFS2 format using the -O option. In this commit UFS1 is the default format, so if you want to build UFS2 format filesystems, you must specify -O 2. This default will be changed to UFS2 when UFS2 proves itself to be stable. In this commit the boot code for reading UFS2 filesystems is not compiled (see /sys/boot/common/ufsread.c) as there is insufficient space in the boot block. Once the size of the boot block is increased, this code can be defined. Things to note: the definition of SBSIZE has changed to SBLOCKSIZE. The header file <ufs/ufs/dinode.h> must be included before <ufs/ffs/fs.h> so as to get the definitions of ufs2_daddr_t and ufs_lbn_t. Still TODO: Verify that the first level bootstraps work for all the architectures. Convert the utility ffsinfo to understand UFS2 and test growfs. Add support for the extended attribute storage. Update soft updates to ensure integrity of extended attribute storage. Switch the current extended attribute interfaces to use the extended attribute storage. Add the extent like functionality (framework is there, but is currently never used). Sponsored by: DARPA & NAI Labs. Reviewed by: Poul-Henning Kamp <phk@freebsd.org>
2002-06-21 07:18:05 +01:00
*
Further cleanups.
2002-03-19 20:01:38 +00:00
* Copyright (c) 1980, 1989, 1993
* The Regents of the University of California. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
Renumber copyright clause 4 Renumber cluase 4 to 3, per what everybody else did when BSD granted them permission to remove clause 3. My insistance on keeping the same numbering for legal reasons is too pedantic, so give up on that point. Submitted by: Jan Schaumann <jschauma@stevens.edu> Pull Request: https://github.com/freebsd/freebsd/pull/96
2017-02-28 23:42:47 +00:00
* 3. Neither the name of the University nor the names of its contributors
Further cleanups.
2002-03-19 20:01:38 +00:00
* may be used to endorse or promote products derived from this software
* without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
* SUCH DAMAGE.
*/
Convert newfs to libufs (really). Solves one real issue with previous version of such. Differences in filesystems generated were found to be from 1) sbwrite with the "all" parameter 2) removal of writecache. The sbwrite call was made to perform as the original version, and otherwise this was checked against a version of newfs with the write cache removed.
2003-02-11 03:06:45 +00:00
#include <libufs.h>
Further cleanups.
2002-03-19 20:01:38 +00:00
Ensure that newfs will never create a filesystem with more than 2^32 inodes by cutting back on the number of inodes per cylinder group if necessary to stay under the limit. For a default (16K block) file system, this limit begins to take effect for file systems above 32Tb. This fix is in addition to -r203763 which corrected a problem in the kernel that treated large inode numbers as negative rather than unsigned. For a default (16K block) file system, this bug began to show up at a file system size above about 16Tb. Reported by: Scott Burns, John Kilburg, Bruce Evans Followup by: Jeff Roberson PR: 133980 MFC after: 2 weeks
2010-02-10 20:17:46 +00:00
/*
* The following two constants set the default block and fragment sizes.
* Both constants must be a power of 2 and meet the following constraints:
* MINBSIZE <= DESBLKSIZE <= MAXBSIZE
* sectorsize <= DESFRAGSIZE <= DESBLKSIZE
* DESBLKSIZE / DESFRAGSIZE <= 8
*/
Raise the default blocksize for UFS/FFS filesystems from 16K to 32K and the default fragment size from 2K to 4K. The rational is that most disks are now running with 4K sectors. While they can (slowly) simulate 512-byte sectors by doing a read-modify-write, it is desirable to avoid this functionality. By raising the minimum filesystem allocation to 4K, the filesystem will never trigger the small sector emulation. Also, the growth of disk sizes has lead us to double the default block size about every ten years. The rise from 8K to 16K blocks was done in 2001. So, by the 10-year metric, the time has come for 32K blocks. Discussed at: May 2011 BSDCan Developer Summit Reference: http://wiki.freebsd.org/201105DevSummit/FileSystems
2011-05-26 19:22:49 +01:00
#define DFL_FRAGSIZE 4096
#define DFL_BLKSIZE 32768
Ensure that newfs will never create a filesystem with more than 2^32 inodes by cutting back on the number of inodes per cylinder group if necessary to stay under the limit. For a default (16K block) file system, this limit begins to take effect for file systems above 32Tb. This fix is in addition to -r203763 which corrected a problem in the kernel that treated large inode numbers as negative rather than unsigned. For a default (16K block) file system, this bug began to show up at a file system size above about 16Tb. Reported by: Scott Burns, John Kilburg, Bruce Evans Followup by: Jeff Roberson PR: 133980 MFC after: 2 weeks
2010-02-10 20:17:46 +00:00
/*
* Cylinder groups may have up to MAXBLKSPERCG blocks. The actual
* number used depends upon how much information can be stored
* in a cylinder group map which must fit in a single file system
* block. The default is to use as many as possible blocks per group.
*/
#define MAXBLKSPERCG 0x7fffffff /* desired fs_fpg ("infinity") */
/*
* MAXBLKPG determines the maximum number of data blocks which are
* placed in a single cylinder group. The default is one indirect
* block worth of data blocks.
*/
#define MAXBLKPG(bsize) ((bsize) / sizeof(ufs2_daddr_t))
/*
* Each file system has a number of inodes statically allocated.
* We allocate one inode slot per NFPI fragments, expecting this
* to be far more than we will ever need.
*/
Reduce NFPI by half to keep the default number of inodes the same with the now default 32K/4K filesystem the same as the number of inodes in the previously default 16K/2K filesystem. PR: bin/16265 Reported by: Olivier Cochard-Labbe <olivier@cochard.me> MFC: 4 weeks (9 is the only affected branch)
2011-12-21 22:49:16 +00:00
#define NFPI 2
Ensure that newfs will never create a filesystem with more than 2^32 inodes by cutting back on the number of inodes per cylinder group if necessary to stay under the limit. For a default (16K block) file system, this limit begins to take effect for file systems above 32Tb. This fix is in addition to -r203763 which corrected a problem in the kernel that treated large inode numbers as negative rather than unsigned. For a default (16K block) file system, this bug began to show up at a file system size above about 16Tb. Reported by: Scott Burns, John Kilburg, Bruce Evans Followup by: Jeff Roberson PR: 133980 MFC after: 2 weeks
2010-02-10 20:17:46 +00:00
Further cleanups.
2002-03-19 20:01:38 +00:00
/*
* variables set up by front end.
*/
Rename the undocumented -E option to -X. Implement -E option which will erase the filesystem sectors before making the new filesystem. Reserved space in front of the superblock (bootcode) is not erased. NB: Erasing can take as long time as writing every sector sequentially. This is relevant for all flash based disks which use wearlevelling.
2007-12-16 19:41:31 +00:00
extern int Eflag; /* Erase previous disk contents */
Bring in support for volume labels to the filesystem utilities. Reviewed by: mckusick
2003-02-01 04:17:10 +00:00
extern int Lflag; /* add a volume label */
s/filesystem/file system/g as discussed on -developers
2002-08-21 19:11:48 +01:00
extern int Nflag; /* run mkfs without writing file system */
extern int Oflag; /* build UFS1 format file system */
Add the undocumented -R option to disable randomness for regression-testing. Add a couple of simple regression tests accessible with "make test", they depend on the md(4) driver. FYI I have also tried running the test against a week old newfs and it passed.
2002-03-19 21:05:29 +00:00
extern int Rflag; /* regression test */
s/filesystem/file system/g as discussed on -developers
2002-08-21 19:11:48 +01:00
extern int Uflag; /* enable soft updates for file system */
Add missing static keywords for global variables to tools in sbin/. These tools declare global variables without using the static keyword, even though their use is limited to a single C-file, or without placing an extern declaration of them in the proper header file.
2011-11-04 13:36:02 +00:00
extern int jflag; /* enable soft updates journaling for filesys */
Rename the undocumented -E option to -X. Implement -E option which will erase the filesystem sectors before making the new filesystem. Reserved space in front of the superblock (bootcode) is not erased. NB: Erasing can take as long time as writing every sector sequentially. This is relevant for all flash based disks which use wearlevelling.
2007-12-16 19:41:31 +00:00
extern int Xflag; /* exit in middle of newfs for testing */
Add -J flag to both newfs(8) and tunefs(8) which allows to enable gjournal support. I left -j flag for UFS journal implementation which we may gain at some point. Sponsored by: home.pl
2006-10-31 21:52:28 +00:00
extern int Jflag; /* enable gjournal for file system */
Add a "-l" flag to newfs, which sets the FS_MULTILABEL flag. This permits users of newfs to set the multilabel flag on UFS1 and UFS2 file systems from inception without using tunefs. Obtained from: TrustedBSD Project Sponsored by: DARPA, McAfee Research
2004-02-26 01:14:27 +00:00
extern int lflag; /* enable multilabel MAC for file system */
Add an option to suppress the creation of the .snap directory in the new filesystem. This is intended for memory and vnode filesystems that will never be fsck'ed or dumped. Obtained from: St. Bernard Software RAPID MFC after: 2 weeks
2005-01-21 22:20:25 +00:00
extern int nflag; /* do not create .snap directory */
Add support for FS_TRIM to user-mode UFS utilities. Reviewed by: mckusick, pjd, pho Tested by: pho MFC after: 1 month
2010-12-29 12:31:18 +00:00
extern int tflag; /* enable TRIM */
MFp4: Add a new option to newfs(8), -r, to specify reserved space at the end of the device. It can be useful, e.g., when the device is to become a member of a gmirror array later w/o losing the file system on it. Document the new option in the manpage. While I'm here, improve error handling for -s option, which is syntactically similar to -r; and document the fact that -s0 selects the default fs size explicitly, which can be useful, e.g., in a menu-based wrapper around newfs(8) requiring some value be entered for the fs size. Also fix a small typo in the help line for -s (missing space). Idea and initial implementation by: marck Discussed on: -fs Critical review by: bde Tested with: cmp(1)
2007-11-28 07:29:10 +00:00
extern intmax_t fssize; /* file system size */
Fix problem with geom_label(4) not recognizing UFS labels on filesystems extended using growfs(8). The problem here is that geom_label checks if the filesystem size recorded in UFS superblock is equal to the provider (i.e. device) size. This check cannot be removed due to backward compatibility. On the other hand, in most cases growfs(8) cannot set fs_size in the superblock to match the provider size, because, differently from newfs(8), it cannot recompute cylinder group sizes. To fix this problem, add another superblock field, fs_providersize, used only for this purpose. The geom_label(4) will attach if either fs_size (filesystem created with newfs(8)) or fs_providersize (filesystem expanded using growfs(8)) matches the device size. PR: kern/165962 Reviewed by: mckusick Sponsored by: FreeBSD Foundation
2012-10-30 21:32:10 +00:00
extern off_t mediasize; /* device size */
Change secrorsize back to int, since that's the data type expected by the ioctl(DIOCGSECTORSIZE). It creates issues on some architectures. MFC after: 1 week Reported by: Jayachandran C.
2010-03-09 10:31:03 +00:00
extern int sectorsize; /* bytes/sector */
Further cleanups.
2002-03-19 20:01:38 +00:00
extern int realsectorsize; /* bytes/sector in hardware*/
o bdeficize expand_number_int() function; o revert most of the recent changes (int -> int64_t conversion) by using this functon for parsing all options.
2010-03-09 19:31:08 +00:00
extern int fsize; /* fragment size */
extern int bsize; /* block size */
extern int maxbsize; /* maximum clustering */
extern int maxblkspercg; /* maximum blocks per cylinder group */
Further cleanups.
2002-03-19 20:01:38 +00:00
extern int minfree; /* free space threshold */
The purpose of this change to the FFS layout policy is to reduce the running time for a full fsck. It also reduces the random access time for large files and speeds the traversal time for directory tree walks. The key idea is to reserve a small area in each cylinder group immediately following the inode blocks for the use of metadata, specifically indirect blocks and directory contents. The new policy is to preferentially place metadata in the metadata area and everything else in the blocks that follow the metadata area. The size of this area can be set when creating a filesystem using newfs(8) or changed in an existing filesystem using tunefs(8). Both utilities use the `-k held-for-metadata-blocks' option to specify the amount of space to be held for metadata blocks in each cylinder group. By default, newfs(8) sets this area to half of minfree (typically 4% of the data area). This work was inspired by a paper presented at Usenix's FAST '13: www.usenix.org/conference/fast13/ffsck-fast-file-system-checker Details of this implementation appears in the April 2013 of ;login: www.usenix.org/publications/login/april-2013-volume-38-number-2. A copy of the April 2013 ;login: paper can also be downloaded from: www.mckusick.com/publications/faster_fsck.pdf. Reviewed by: kib Tested by: Peter Holm MFC after: 4 weeks
2013-03-22 21:45:28 +00:00
extern int metaspace; /* space held for metadata blocks */
Further cleanups.
2002-03-19 20:01:38 +00:00
extern int opt; /* optimization preference (space or time) */
o bdeficize expand_number_int() function; o revert most of the recent changes (int -> int64_t conversion) by using this functon for parsing all options.
2010-03-09 19:31:08 +00:00
extern int density; /* number of bytes per inode */
extern int maxcontig; /* max contiguous blocks to allocate */
extern int maxbpg; /* maximum blocks per file in a cyl group */
extern int avgfilesize; /* expected average file size */
extern int avgfilesperdir; /* expected number of files per directory */
Bring in support for volume labels to the filesystem utilities. Reviewed by: mckusick
2003-02-01 04:17:10 +00:00
extern u_char *volumelabel; /* volume label for filesystem */
Convert newfs to libufs (really). Solves one real issue with previous version of such. Differences in filesystems generated were found to be from 1) sbwrite with the "all" parameter 2) removal of writecache. The sbwrite call was made to perform as the original version, and otherwise this was checked against a version of newfs with the write cache removed.
2003-02-11 03:06:45 +00:00
extern struct uufsd disk; /* libufs disk structure */
Continue the cleanup preparations for UFS2 (& GEOM): Use only one filedescriptor. Open in R/O or R/W based in the '-N' option. Make the filedescriptor a global variable instead of passing it around as semi-global variable(s). Remove the undocumented ability to specify type without '-T' option. Replace fatal() with straight err(3)/errx(3). Save calls to strerror() where applicable. Loose the progname variable. Get the sense of the cpgflag test correct so we only issue warnings if people specify cpg and can't get that. It can be argued that this should be an error. Remove the check to see if the disk is mounted: Open for writing would fail if it were mounted. Attempt to get the sectorsize and mediasize with the generic disk ioctls, fall back to disklabel and /etc/disktab as we can. Notice that on-disk labels still take precedence over /etc/disktab, this is probably wrong, but not as wrong as the entire concept of /etc/disktab is. Sponsored by: DARPA & NAI Labs.
2002-04-24 12:44:02 +01:00
Enable operation of newfs on plain files, which is useful when you want to prepare disk images for emulators (though 'makefs' in port can do something similar). This relies on: + minor changes to pass the consistency checks even when working on a file; + an additional option, '-p partition' , to specify the disk partition to initialize; + some changes on the I/O routines to deal with partition offsets. The latter was a bit tricky to implement, see the details in newfs.h: in newfs, I/O is done through libufs which assumes that the file descriptor refers to the whole partition. Introducing support for the offset in libufs would require a non-backward compatible change in the library, to be dealt with a version bump or with symbol versioning. I felt both approaches to be overkill for this specific application, especially because there might be other changes to libufs that might become necessary in the near future. So I used the following trick: - read access is always done by calling bread() directly, so we just add the offset in the (few) places that call bread(); - write access is done through bwrite() and sbwrite(), which in turn calls bwrite(). To avoid rewriting sbwrite(), we supply our own version of bwrite() here, which takes precedence over the version in libufs. MFC after: 4 weeks
2008-12-03 18:36:59 +00:00
/*
* To override a limitation in libufs, export the offset (in sectors) of the
* partition on the underlying media (file or disk). The value is used as
* an offset for all accesses to the media through bread(), which is only
* invoked directly in this program.
* For bwrite() we need a different approach, namely override the library
* version with one defined here. This is because bwrite() is called also
* by the library function sbwrite() which we cannot intercept nor want to
* rewrite. As a consequence, the internal version of bwrite() adds the
* partition offset itself when calling the underlying function, pwrite().
*
* XXX This info really ought to go into the struct uufsd, at which point
* we can remove the above hack.
*/
extern ufs2_daddr_t part_ofs; /* partition offset in blocks */
Continue the cleanup preparations for UFS2 (& GEOM): Use only one filedescriptor. Open in R/O or R/W based in the '-N' option. Make the filedescriptor a global variable instead of passing it around as semi-global variable(s). Remove the undocumented ability to specify type without '-T' option. Replace fatal() with straight err(3)/errx(3). Save calls to strerror() where applicable. Loose the progname variable. Get the sense of the cpgflag test correct so we only issue warnings if people specify cpg and can't get that. It can be argued that this should be an error. Remove the check to see if the disk is mounted: Open for writing would fail if it were mounted. Attempt to get the sectorsize and mediasize with the generic disk ioctls, fall back to disklabel and /etc/disktab as we can. Notice that on-disk labels still take precedence over /etc/disktab, this is probably wrong, but not as wrong as the entire concept of /etc/disktab is. Sponsored by: DARPA & NAI Labs.
2002-04-24 12:44:02 +01:00
void mkfs (struct partition *, char *);
Reference in New Issue Copy Permalink