freebsd-src/release/Makefile.ec2

60 lines
1.8 KiB
Makefile
Raw Normal View History

MFC r280299, r280840, r280846, r280879, r280881, r280884, r280928, r281783, r281809, r281832: r280299 (cperciva): When creating VM images, copy the contents of the created filesystem into a new filesystem before packaging it into a disk image. This prevents "remnants" of deleted files from showing up in the VM images, and reduces their compressed size (by about 10% for the cloudware images) as a result. r280840 (cperciva): Clean up filesystem unmounting in vmimage builds: - Remove vm_umount_base function which is currently unused. - Add umount_loop function which loops attempting to unmount one filesystem. - Replace calls to umount with calls to umount_loop. - Don't attempt to unmount ${DESTDIR}/dev if it isn't mounted. The looping is necessary because sometimes umount fails due to filesystems being busy. The most common cause of such busyness is periodic(8) jobs running `find / ...`. r280846 (cperciva): Improve check for whether ${DESTDIR}/dev is mounted. r280879 (cperciva): Add bits for building EC2 disk images. r280881 (cperciva): Partially revert r278118 now that the required logic for deciding whether freebsd-update can be useful has moved into the firstboot_freebsd_update script. r280884 (cperciva): Fix pkg(8) command line in vm_extra_pre_umount(). r280928 (cperciva): Add code for creating an EC2 AMI. r281783: When building VM disk images, vm_copy_base() uses tar(1) to copy the userland from one md(4)-mounted filesystem to a clean filesystem to prevent remnants of files that were added and removed from resulting in an unclean filesystem. When newfs(8) creates the first filesystem with journaled soft-updates enabled, the /.sujournal file in the new filesystem cannot be overwritten by the /.sujournal in the original filesystem. To avoid this particular error case, do not enable journaled soft-updates when creating the md(4)-backed filesystems, and instead use tunefs(8) to enable journaled soft-updates after the new filesystem is populated in vm_copy_base(). While here, fix a long standing bug where the build environment /boot files were used by mkimg(1) when creating the VM disk images by using the files in .OBJDIR. r281809: Simplify variable expansion in attempt to fix the vm-image build. r281832: Fix locating the /boot files. Sponsored by: The FreeBSD Foundation
2015-04-27 20:41:40 +01:00
#
# $FreeBSD$
#
#
# Makefile for creating an EC2 AMI from a disk image.
#
.if ${BRANCH} == "CURRENT" || ${BRANCH} == "STABLE" || ${BRANCH} == "PRERELEASE"
MFC r284893, r284895-r284897, r284942, r284968, r284996, r285005: r284893 (brd): Add initial support for building Vagrant images for VMWare. Next steps will be adding Virtualbox support and uploading to Hashicorp Atlas for others to consume. r284895: Add default VAGRANT_IMG variable. r284896: Remove _ACCOUNT and add _USERNAME, _NAME, _VERSION for the VAGRANT_${VAR} variables extracted from VAGRANT_UPLOAD_CONF. Set ATLAS_${VAR} to VAGRANT_${VAR} if VAGRANT_UPLOAD_CONF is set. There is intent to intentionally have separate variants of configuration entries, but the defaults do not yet have any reason to be different. r284897: Instead of hard-coding the PROVIDERS for upload, add the VAGRANT_PROVIDERS variable. Right now, it defaults to only vmware_desktop, virtualbox support is to follow at some point. While here, fix the hashicorp URL: s/vagrant/atlas/, which was result of a sed(1) replace (and my fault). r284942 (brd): Add Support for uploading Vagrant images to Hashicorp Atlas. r284968: Default the VAGRANT_VERSION to ${REVISION}-${BRANCH} if not set, which expands to '11.0-CURRENT', for example. If the branch is -CURRENT, -STABLE, or -PRERELEASE, suffix the VAGRANT_VERSION with the snapshot date. r284996: Fix the gcloud port/package name. r285005: Remove the HH-MM suffix from the build date suffix. It was useful when working out several kinks when testing automated image uploading when retrying was necessary, but now it is making things much too messy. Sponsored by: The FreeBSD Foundation
2015-07-02 17:17:05 +01:00
AMINAMESUFFIX!= date +-%Y-%m-%d
MFC r280299, r280840, r280846, r280879, r280881, r280884, r280928, r281783, r281809, r281832: r280299 (cperciva): When creating VM images, copy the contents of the created filesystem into a new filesystem before packaging it into a disk image. This prevents "remnants" of deleted files from showing up in the VM images, and reduces their compressed size (by about 10% for the cloudware images) as a result. r280840 (cperciva): Clean up filesystem unmounting in vmimage builds: - Remove vm_umount_base function which is currently unused. - Add umount_loop function which loops attempting to unmount one filesystem. - Replace calls to umount with calls to umount_loop. - Don't attempt to unmount ${DESTDIR}/dev if it isn't mounted. The looping is necessary because sometimes umount fails due to filesystems being busy. The most common cause of such busyness is periodic(8) jobs running `find / ...`. r280846 (cperciva): Improve check for whether ${DESTDIR}/dev is mounted. r280879 (cperciva): Add bits for building EC2 disk images. r280881 (cperciva): Partially revert r278118 now that the required logic for deciding whether freebsd-update can be useful has moved into the firstboot_freebsd_update script. r280884 (cperciva): Fix pkg(8) command line in vm_extra_pre_umount(). r280928 (cperciva): Add code for creating an EC2 AMI. r281783: When building VM disk images, vm_copy_base() uses tar(1) to copy the userland from one md(4)-mounted filesystem to a clean filesystem to prevent remnants of files that were added and removed from resulting in an unclean filesystem. When newfs(8) creates the first filesystem with journaled soft-updates enabled, the /.sujournal file in the new filesystem cannot be overwritten by the /.sujournal in the original filesystem. To avoid this particular error case, do not enable journaled soft-updates when creating the md(4)-backed filesystems, and instead use tunefs(8) to enable journaled soft-updates after the new filesystem is populated in vm_copy_base(). While here, fix a long standing bug where the build environment /boot files were used by mkimg(1) when creating the VM disk images by using the files in .OBJDIR. r281809: Simplify variable expansion in attempt to fix the vm-image build. r281832: Fix locating the /boot files. Sponsored by: The FreeBSD Foundation
2015-04-27 20:41:40 +01:00
.endif
.if defined(EC2PUBLIC)
PUBLISH= --public
.endif
CLEANFILES+= ec2ami
.if !exists(/usr/local/bin/bsdec2-image-upload)
CW_EC2_PORTINSTALL= cw-ec2-portinstall
CLEANFILES+= ${CW_EC2_PORTINSTALL}
.else
CW_EC2_PORTINSTALL=
.endif
cw-ec2-portinstall:
.if exists(${PORTSDIR}/net/bsdec2-image-upload/Makefile)
make -C ${PORTSDIR}/net/bsdec2-image-upload BATCH=1 all install clean
.else
. if !exists(/usr/local/sbin/pkg-static)
env ASSUME_ALWAYS_YES=yes pkg bootstrap -y
. endif
env ASSUME_ALWAYS_YES=yes pkg install -y net/bsdec2-image-upload
MFC r280299, r280840, r280846, r280879, r280881, r280884, r280928, r281783, r281809, r281832: r280299 (cperciva): When creating VM images, copy the contents of the created filesystem into a new filesystem before packaging it into a disk image. This prevents "remnants" of deleted files from showing up in the VM images, and reduces their compressed size (by about 10% for the cloudware images) as a result. r280840 (cperciva): Clean up filesystem unmounting in vmimage builds: - Remove vm_umount_base function which is currently unused. - Add umount_loop function which loops attempting to unmount one filesystem. - Replace calls to umount with calls to umount_loop. - Don't attempt to unmount ${DESTDIR}/dev if it isn't mounted. The looping is necessary because sometimes umount fails due to filesystems being busy. The most common cause of such busyness is periodic(8) jobs running `find / ...`. r280846 (cperciva): Improve check for whether ${DESTDIR}/dev is mounted. r280879 (cperciva): Add bits for building EC2 disk images. r280881 (cperciva): Partially revert r278118 now that the required logic for deciding whether freebsd-update can be useful has moved into the firstboot_freebsd_update script. r280884 (cperciva): Fix pkg(8) command line in vm_extra_pre_umount(). r280928 (cperciva): Add code for creating an EC2 AMI. r281783: When building VM disk images, vm_copy_base() uses tar(1) to copy the userland from one md(4)-mounted filesystem to a clean filesystem to prevent remnants of files that were added and removed from resulting in an unclean filesystem. When newfs(8) creates the first filesystem with journaled soft-updates enabled, the /.sujournal file in the new filesystem cannot be overwritten by the /.sujournal in the original filesystem. To avoid this particular error case, do not enable journaled soft-updates when creating the md(4)-backed filesystems, and instead use tunefs(8) to enable journaled soft-updates after the new filesystem is populated in vm_copy_base(). While here, fix a long standing bug where the build environment /boot files were used by mkimg(1) when creating the VM disk images by using the files in .OBJDIR. r281809: Simplify variable expansion in attempt to fix the vm-image build. r281832: Fix locating the /boot files. Sponsored by: The FreeBSD Foundation
2015-04-27 20:41:40 +01:00
.endif
@touch ${.TARGET}
ec2ami: cw-ec2 ${CW_EC2_PORTINSTALL}
MFC r280299, r280840, r280846, r280879, r280881, r280884, r280928, r281783, r281809, r281832: r280299 (cperciva): When creating VM images, copy the contents of the created filesystem into a new filesystem before packaging it into a disk image. This prevents "remnants" of deleted files from showing up in the VM images, and reduces their compressed size (by about 10% for the cloudware images) as a result. r280840 (cperciva): Clean up filesystem unmounting in vmimage builds: - Remove vm_umount_base function which is currently unused. - Add umount_loop function which loops attempting to unmount one filesystem. - Replace calls to umount with calls to umount_loop. - Don't attempt to unmount ${DESTDIR}/dev if it isn't mounted. The looping is necessary because sometimes umount fails due to filesystems being busy. The most common cause of such busyness is periodic(8) jobs running `find / ...`. r280846 (cperciva): Improve check for whether ${DESTDIR}/dev is mounted. r280879 (cperciva): Add bits for building EC2 disk images. r280881 (cperciva): Partially revert r278118 now that the required logic for deciding whether freebsd-update can be useful has moved into the firstboot_freebsd_update script. r280884 (cperciva): Fix pkg(8) command line in vm_extra_pre_umount(). r280928 (cperciva): Add code for creating an EC2 AMI. r281783: When building VM disk images, vm_copy_base() uses tar(1) to copy the userland from one md(4)-mounted filesystem to a clean filesystem to prevent remnants of files that were added and removed from resulting in an unclean filesystem. When newfs(8) creates the first filesystem with journaled soft-updates enabled, the /.sujournal file in the new filesystem cannot be overwritten by the /.sujournal in the original filesystem. To avoid this particular error case, do not enable journaled soft-updates when creating the md(4)-backed filesystems, and instead use tunefs(8) to enable journaled soft-updates after the new filesystem is populated in vm_copy_base(). While here, fix a long standing bug where the build environment /boot files were used by mkimg(1) when creating the VM disk images by using the files in .OBJDIR. r281809: Simplify variable expansion in attempt to fix the vm-image build. r281832: Fix locating the /boot files. Sponsored by: The FreeBSD Foundation
2015-04-27 20:41:40 +01:00
.if !defined(AWSKEYFILE) || !exists(${AWSKEYFILE})
@echo "--------------------------------------------------------------"
@echo ">>> AWSKEYFILE must point at AWS keys for EC2 AMI creation"
@echo "--------------------------------------------------------------"
@false
.endif
.if !defined(AWSREGION)
@echo "--------------------------------------------------------------"
@echo ">>> AWSREGION must be specified EC2 AMI creation"
@echo "--------------------------------------------------------------"
@false
.endif
.if !defined(AWSBUCKET)
@echo "--------------------------------------------------------------"
@echo ">>> AWSBUCKET must be specified for EC2 AMI creation"
@echo "--------------------------------------------------------------"
@false
.endif
/usr/local/bin/bsdec2-image-upload ${PUBLISH} \
${.OBJDIR}/ec2.raw \
"${TYPE} ${REVISION}-${BRANCH}-${TARGET}${AMINAMESUFFIX}" \
"${TYPE} ${REVISION}-${BRANCH}-${TARGET}" \
MFC r280299, r280840, r280846, r280879, r280881, r280884, r280928, r281783, r281809, r281832: r280299 (cperciva): When creating VM images, copy the contents of the created filesystem into a new filesystem before packaging it into a disk image. This prevents "remnants" of deleted files from showing up in the VM images, and reduces their compressed size (by about 10% for the cloudware images) as a result. r280840 (cperciva): Clean up filesystem unmounting in vmimage builds: - Remove vm_umount_base function which is currently unused. - Add umount_loop function which loops attempting to unmount one filesystem. - Replace calls to umount with calls to umount_loop. - Don't attempt to unmount ${DESTDIR}/dev if it isn't mounted. The looping is necessary because sometimes umount fails due to filesystems being busy. The most common cause of such busyness is periodic(8) jobs running `find / ...`. r280846 (cperciva): Improve check for whether ${DESTDIR}/dev is mounted. r280879 (cperciva): Add bits for building EC2 disk images. r280881 (cperciva): Partially revert r278118 now that the required logic for deciding whether freebsd-update can be useful has moved into the firstboot_freebsd_update script. r280884 (cperciva): Fix pkg(8) command line in vm_extra_pre_umount(). r280928 (cperciva): Add code for creating an EC2 AMI. r281783: When building VM disk images, vm_copy_base() uses tar(1) to copy the userland from one md(4)-mounted filesystem to a clean filesystem to prevent remnants of files that were added and removed from resulting in an unclean filesystem. When newfs(8) creates the first filesystem with journaled soft-updates enabled, the /.sujournal file in the new filesystem cannot be overwritten by the /.sujournal in the original filesystem. To avoid this particular error case, do not enable journaled soft-updates when creating the md(4)-backed filesystems, and instead use tunefs(8) to enable journaled soft-updates after the new filesystem is populated in vm_copy_base(). While here, fix a long standing bug where the build environment /boot files were used by mkimg(1) when creating the VM disk images by using the files in .OBJDIR. r281809: Simplify variable expansion in attempt to fix the vm-image build. r281832: Fix locating the /boot files. Sponsored by: The FreeBSD Foundation
2015-04-27 20:41:40 +01:00
${AWSREGION} ${AWSBUCKET} ${AWSKEYFILE}
@touch ${.TARGET}