diff --git a/CHANGELOG.md b/CHANGELOG.md index 393f1e9..1aca8ff 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,18 +2,46 @@ ## 2023-05-XX - Tiny Cloud v3.0.0 +### INIT SCRIPTS / PHASES + +* Tiny Cloud has been reorganized into **four** init script phases... + * `boot` - "boot" runlevel, after `syslogd` is started, and before + networking is up. + * `early` - "default" runlevel, after networking is up, but *before* + user-data is made available. This phase is responsible for pulling in + user-data for *later* phases. + * `main` - "default" runlevel, after user-data is available. Most things + get done here. + * `final` - "default" runlevel, after all other init scripts have finished. + Typically user-data scripts are run here, and bootstrap status is marked + as complete. + * Tiny Cloud init functionality has been consolidated into **/sbin/tiny-cloud** - and init scripts should use `tiny-cloud ` to indicate whether `early`, - `main`, or `final` actions should be taken. Additionally, it is now possible - for clouds to specify their own (or supercede the default) init functions - and/or change which init phase they are executed in. + and init scripts should use `tiny-cloud ` to indicate whether `boot`, + `early`, `main`, or `final` actions should be taken. + +* Use `tiny-cloud --setup` to add/update Tiny Cloud's init scripts into the + right runlevels. Currently only OpenRC is supported. * The example OpenRC init scripts been updated and moved to **dist/openrc/**. - Use `tiny-cloud --setup` to add Tiny Cloud's init scripts into the right - runlevels +### NEW STUFF -* Tiny Cloud configuration has moved to **/etc/tiny-cloud.conf**. +* Tiny Cloud now supports the concept of user-data handlers, to support acting + on different payload content-types. + +* Clouds and user-data handlers can specify their own (or supercede the default) + init functions and/or change the init phase in which they are executed. The + order of declaration is default --> cloud --> user-data (last one wins). + +* We now have an experimental `alpine` installer "cloud", based on NoCloud, + and an associated `#alpine-config` user-data handler, which supports a subset + of `#cloud-config` features, plus some extensions. As this is new, we expect + it to experience some continued evolution. + +* Also thanks to Nataniel Copa (@ncopa), Tiny Cloud now has unit tests. + +### DEPRECATION * `nvme-ebs-symlinks` has been _deprecated_ and disabled by default. The **mdev-conf** package, as of v4.4 is now responsible for maintaining NVMe @@ -23,6 +51,10 @@ **/dev/sd** or **/dev/xvd** symlinks are created as indicated in NVMe device metadata, *but NOT both*! +### MISCELLANEOUS + +* Tiny Cloud configuration has moved to **/etc/tiny-cloud.conf**. + * `imds` now supports `@local-hostname` alias. For most clouds this is the same as `@hostname`. diff --git a/README.md b/README.md index 86a5e75..77357a2 100644 --- a/README.md +++ b/README.md @@ -8,57 +8,64 @@ do just what is necessary with a small footprint and minimal dependencies. A direct descendant of [tiny-ec2-bootstrap]( https://gitlab.alpinelinux.org/alpine/cloud/tiny-ec2-bootstrap), Tiny Cloud works with multiple cloud providers. Currently, the following are supported: -* [AWS](https://aws.amazon.com): Amazon Web Services -* [Azure](https://azure.microsoft.com): Microsoft Azure -* [GCP](https://cloud.google.com): Google Cloud Platform -* [OCI](https://cloud.oracle.com): Oracle Cloud Infrastructure +* [AWS](https://aws.amazon.com) - Amazon Web Services +* [Azure](https://azure.microsoft.com) - Microsoft Azure +* [GCP](https://cloud.google.com) - Google Cloud Platform +* [OCI](https://cloud.oracle.com) - Oracle Cloud Infrastructure * [NoCloud]( - https://cloudinit.readthedocs.io/en/latest/reference/datasources/nocloud.html - ): - cloud-init's NoCloud AWS-compatible user provided data source + https://cloudinit.readthedocs.io/en/latest/reference/datasources/nocloud.html + ) - cloud-init's NoCloud AWS-compatible user provided data source + +Tiny Cloud is also used for Alpine Linux's experimental "auto-install" feature. ## Features The following actions will occur ***only once***, during the initial boot of an instance: -* expand the root filesystem to use all available root device space, during the - **sysinit** runlevel -* set the instance's hostname from instance metadata -* install SSH keys from instance metadata to the cloud user account's - `authorized_keys` file (the user must already exist) -* save the instance user-data to a file, and if it's a script, execute it at - the end of the **default** runlevel +* expand the root filesystem to use all available root device space +* set up default network interfaces, if necessary +* enable `sshd`, if necessary +* save instance user-data to a file, decompress if necessary +* create default cloud user, if necessary +* set the instance's hostname from instance meta-data +* install SSH keys from instance meta-data to the cloud user account's + `authorized_keys` file +* if instance user-data is a script, execute it at the end of the **default** + runlevel +* mark the bootstrap of the instance as "complete" Optional features, which may not be universally necessary: -* manage hotpluggable network interfaces -* sync IMDS-provided secondary IPv4 and IPv6 addresses network interfaces -* manage symlinks from NVMe block devices to `/dev/xvd` or `/dev/sd` devices - (i.e. AWS Nitro instances) +* manage hotpluggable virtual network interfaces +* sync IMDS-provided secondary IPv4 and IPv6 network configuration + +Other cloud- and user-data-specific actions may also occur. Also included is a handy `imds` client script for easy access to an instance's IMDS data. ## Requirements -As Tiny Cloud is meant to be tiny, it has very few dependencies: +As Tiny Cloud is meant to be tiny, it has few dependencies: * Busybox (`ash`, `wget`, etc.) -* `ifupdown-ng` (optional, for network management) -* `iproute2-minimal` (optional, for syncing IPv4/IPv6 from IMDS) -* `nvme-cli` (optional, for AWS nitro NVMe symlinks) +* `e2fsprogs-extra` (for `resize2fs`) +* `openssh-server` * `partx` -* `resize2fs` * `sfdisk` -* [`yx`](https://gitlab.com/tomalok/yx) - (optional, allows NoCloud to extract metadata from YAML files) +* [`yx`](https://gitlab.com/tomalok/yx) (for extracting data from YAML files) -Tiny Cloud has been developed specifically for use with the +Optional dependencies: +* `ifupdown-ng` (for network management) +* `iproute2-minimal` (for syncing IPv4/IPv6 from IMDS) +* `nvme-cli` (for AWS nitro NVMe symlinks) + +_Tiny Cloud has been developed specifically for use with the [Alpine Cloud Images]( https://gitlab.alpinelinux.org/alpine/cloud/alpine-cloud-images) project, and as such, it is currently tailored for use with [Alpine Linux]( https://alpinelinux.org), the [OpenRC](https://github.com/OpenRC/openrc) init system, and the `ext4` root filesystem. If you would like to see Tiny Cloud supported on additional distributions, init systems, and/or filesystems, please -open an issue with your request -- or better yet, submit a merge request! +open an issue with your request -- or better yet, submit a merge request!_ ## Installation @@ -73,13 +80,13 @@ dependencies for Tiny Cloud to support _``_. Alternately, you can download a release tarball, and use `make` to install it. -Next, enable the three primary init scripts... +Next, set up the RC scripts... ``` -rc-update add tiny-cloud-early sysinit -rc-update add tiny-cloud default -rc-update add tiny-cloud-final default +tiny-cloud --setup ``` +That's it! On the next boot, Tiny Cloud will bootstrap the instance. + ## Configuration By default, Tiny Cloud expects configuration at `/etc/tiny-cloud.conf`, @@ -94,40 +101,50 @@ Current valid values are `aws`, `azure`, `gcp`, `oci`, and `nocloud`._ The first time an instance boots -- either freshly instantiated from an image, or after installation on a pre-existing instance -- Tiny Cloud sets up the -instance in three phases... +instance in four phases... + +### Boot Phase + +This phase does not depend on the cloud provider's Instance Meta-Data Service +(IMDS), and does not require networking to be up. `mdev` hotplug modules are +installed (if any), default networking confinguration is set up, `sshd` is +enabled (but not started), and the root partition is expanded. ### Early Phase -The `tiny-cloud-early` init script does not depend on the cloud provider's -Instance MetaData Service (IMDS), and therefore does not have a dependency on -networking. During this "early" phase, the root filesystem is expanded, and -any necessary `mdev` rules for device hotplug are set up. +After networking is up, and the cloud provider's IMDS is available, this phase +is primarily responsible for retrieving the instance's User-Data for use by +later phases. + +User-Data is stored at `/var/lib/cloud/user-data`, and will be decompressed, if +necessary. Currently supported compression algorithms are `gzip`, `bzip2`, +`unxz`, `lzma`, `lzop`, `lz4`, and `zstd`. _(Note that `lz4` and `zstd` are +not installed in Alpine by default, and would need to be added to the image.)_ ### Main Phase -The main `tiny-cloud` init script *does* depend on the cloud provider's IMDS -data; it saves the instance's user data to `/var/lib/cloud/user-data`, sets up -instance's hostname, and installs the cloud user's SSH keys before `sshd` -starts. +When networking, IMDS, and User-Data are all availabile, this is the phase +takes care of the majority of bootstrapping actions that require them -- +setting up the instanxe hostname, creating default cloud user, and installing +SSH keys for it. -If the user data is compressed, Tiny Cloud will decompress it. Currently -supported compression algorithms are `gzip`, `bzip2`, `unxz`, `lzma`, `lzop`, -`lz4`, and `zstd`. _(Note that `lz4` and `zstd` are not installed in Alpine -by default, and would need to be added to the image.)_ +Additional main phase actions may be taken if there is a User-Data handler +defined for its content type, and those actions are associated with the main +phase. ### Final Phase -`tiny-cloud-final` should be the very last init script to run in the -**default** runlevel. +The very last thing to be run in the **default** runlevel this phase will +execute the saved User-Data, if it is a script starting with `#!`; its output +(combined STDOUT and STDER) and exit code are saved to `/var/log/user-data.log` +and `/var/log/user-data.exit`. -If the user data is a script starting with `#!/`, it will be executed; its -output (combined STDOUT and STDERR) and exit code are saved to -`/var/log/user-data.log` and `/var/log/user-data.exit`, respectively; the -directory is overrideable via the `TINY_CLOUD_LOGS` config setting. +Additional final phase actions may be taken if there is a User-Data handler +defined for its content type, and those actions are associated with the final +phase. -If all went well, the very last thing `tiny-cloud-final` does is touch -a `.bootstrap-complete` file into existence in `/var/lib/cloud` or another -directory specified by the `TINY_CLOUD_VAR` config setting. +The very last action to be taken is to mark the instance's bootstrap as +"complete", so that future reboots do not re-boostrap the instance. ### Skipping Init Actions @@ -142,17 +159,19 @@ no-op. To force the init scripts to re-run on the next boot... ``` -rm -f /var/lib/cloud/.bootstrap-complete -``` -or -``` -service tiny-cloud incomplete +tiny-cloud --bootstrap incomplete ``` If you're instantiating an instance in order to create a new cloud image (using [Packer](https://packer.io), or some other means), you will need to do this before creating the image to ensure that instances using the new image will also run Tiny Cloud init scripts during their first boot. +To check the status of the Tiny Cloud bootstrap, use... +``` +tiny-cloud --bootstrap status +``` +...which will either respond with `complete` or `incomplete` + ## Cloud Hotplug Modules ### `vnic_eth_hotplug` @@ -163,12 +182,3 @@ attached/detached from the instance. An `ifupdown-ng` executor also syncs the interfaces' secondary IPv4 and IPV6 addresses associated with those VNICs, if the cloud's IMDS provides that configuration data. - -### `nvme_ebs_links` - -EBS volumes are attached to AWS EC2 Nitro instances using the NVMe driver. -Unfortunately, the `/dev/nvme*` device names do not match the device name -assigned to the attached EBS volume. This hotplug module figures out what the assigned device name is, and sets up `/dev/xvd*` and `/dev/sd*` symlinks to -the right NVMe devices for EBS volumes and their partitions. - -_(deprecated, see [CHANGELOG.md](CHANGELOG.md))_ diff --git a/TODO.md b/TODO.md index 60cf5f1..f921c69 100644 --- a/TODO.md +++ b/TODO.md @@ -1,10 +1,19 @@ # TODO +## SOON-ish + +* Should the extra stuff that the `alpine` installer cloud does also apply to + the `nocloud` cloud? If so, move it there, and the installer is entirely + handled by the user-data handler. + +* Package user-data handlers separately? + +* `#cloud-config` user-data handler (support a useful subset) + + ## FUTURE -* cloud auto-detection - -* `#cloud-config` user-data handler (support a useful subset) +* cloud auto-detection? * `#tiny-config` user-data handler (should be simple-yet-flexible) diff --git a/bin/imds b/bin/imds index 83f7202..7b636ae 100755 --- a/bin/imds +++ b/bin/imds @@ -13,6 +13,7 @@ if [ "$1" = "-h" ] || [ "$1" = "--help" ]; then Usage: imds [-h] { -e | +e | +n | +s | +t | @ | } ... -h : help -e / +e : ignore / catch errors + -E / +E : hide / slow STDERR +n / +s / +t : insert newline / space / tab :- hostname : instance hostname @@ -94,15 +95,17 @@ fi ### non-overrideable functions imds() { - local cmd args key rv err=1 + local cmd args key rv err=1 stderr=1 while [ -n "$1" ]; do cmd=_imds args= key="$1"; shift case $key in - # error handling + # error handling/output -e) err=0; continue ;; # ignore +e) err=1; continue ;; # return + -E) stderr=0; continue ;; # hide + +E) stderr=1; continue ;; # show # TODO: retry/deadline # output control +n) printf "\n"; continue ;; # insert newline @@ -122,8 +125,13 @@ imds() { *) args="$key" ;; esac # TODO: retry/deadline - "$cmd" $args - rv=$? + if [ $stderr = 1 ]; then + "$cmd" $args + rv=$? + else + "$cmd" $args 2>/dev/null + rv=$? + fi [ $err -eq 0 ] && continue [ $rv = "0" ] || return $rv done diff --git a/sbin/imds-net-sync b/sbin/imds-net-sync index 8ae7eae..8bbaa68 100755 --- a/sbin/imds-net-sync +++ b/sbin/imds-net-sync @@ -49,7 +49,7 @@ iface_ip6s() { } imds_ip4s() { - local ip4=$(imds "@nic:$IFACE,@ipv4") + local ip4=$(imds -E "@nic:$IFACE,@ipv4") local ip4s=$(echo "$ip4" | tail +2) # secondary IPv4s local ip4p ip4_cidr ip4_gw @@ -58,7 +58,7 @@ imds_ip4s() { if [ "$IFACE" != eth0 ] && [ -n "$ip4s" ] && [ -z $(ip +F -4 route show table "$RTABLE" 2>/dev/null) ]; then ip4p=$(echo "$ip4" | head -1) # primary IPv4 - ip4_cidr=$(imds "@nic:$IFACE,@ipv4-net") # TODO: get from iface instead? + ip4_cidr=$(imds -E "@nic:$IFACE,@ipv4-net") # TODO: get from iface instead? # TODO: this may not hold true for non-AWS clouds ip4_gw=$(echo "$ip4_cidr" | cut -d/ -f1 | awk -F. '{ print $1"."$2"."$3"."$4+1 }') @@ -73,7 +73,7 @@ imds_ip4s() { # circle back and see how amazon-ec2-net-utils is handling everything these days imds_ip6s() { local ip6s gw tries=20 - ip6s=$(imds "@nic:$IFACE,@ipv6") + ip6s=$(imds -E "@nic:$IFACE,@ipv6") # non-eth0 interfaces need custom route tables # diff --git a/sbin/tiny-cloud b/sbin/tiny-cloud index 4e80a61..e372aac 100755 --- a/sbin/tiny-cloud +++ b/sbin/tiny-cloud @@ -46,13 +46,17 @@ while true; do esac exit 0;; -s|--setup) # just openrc for now - for phase in -boot -early -main -final '' -net; do - rc-update -a del "tiny-cloud$phase" || true + # for installing in mounted volumes + : "${ROOT:=}" + # start with a clean slate + log -s info " - tiny-cloud* services removed from all runlevels" + rm -f "$ROOT"/etc/runlevels/*/tiny-cloud* + log -s info " + tiny-cloud-boot service added to boot runlevel" + ln -s /etc/init.d/tiny-cloud-boot "$ROOT"/etc/runlevel/boot + for p in early main final; do + log -s info " + tiny-cloud-$p service added to default runlevel" + ln -s "/etc/init.d/tiny-cloud-$p" "$ROOT"/etc/runlevel/default done - rc-update add tiny-cloud-boot boot - rc-update add tiny-cloud-early default - rc-update add tiny-cloud-main default - rc-update add tiny-cloud-final default exit 0;; --) shift; break;; *) usage >&2; exit 1;;