Building a Linux host in CLS
Welcome to Campus Linux Services( CLS ).
Here is a quick introduction to the environment and how to join the community as a new tenant: https://youtu.be/kijflGXYoZ0
Working with designated IT staff in your organization
Access to Infoblox, Active Directory and the CLS build environment is generally restricted to departmental or college IT staff. Those who are planning to build a Linux workstation or server for their own use should coordinate that activity with their local IT staff. This will assure that organizational conventions and procedures are followed.
Prerequisites - General
- Account in Infoblox - Used to update host networking and PXE boot information. See Networking at N.C. State under the IPAM subheading for additional information
- Request a domain join user for WolfTech AD (if joining host to Active Directory)
Note: This is only necessary if joining hosts to Active Directory. This configuration is recommended as it allows group-based access control rather than manual lists of users assigned to machines. We do not support group-based access control for machines that authenticate against Wolftech (WOLFTECH.AD.NCSU.EDU) and use OpenLDAP (ldap.ncsu.edu) for user / group information.
Your Windows OU Administrator should be able to get this for you. If not, ping us in #cls and we can help out. The permissions required to join a Linux client to Active Directory are documented at activedirectory.ncsu.edu. - Once you have your service account, follow the directions to set your AD domain join credentials.
Prerequisites - CLS
Request admission as a tenant of the CLS environment by sending an email to LINUX@help.ncsu.edu with :
- Unit name (e.g. "COEDEAN", "CSC", "WOLFPRINT", etc.)Do you have a group in WolfTech AD that can be used to represent your tenant? This will determine who can log under your tenant in our infrastructure.
- Networks / DNS domains of your hosts? These need to be populated in Foreman, if not already there.
- Location of control repository? (see next steps) (e.g. git@github.ncsu.edu:organization/control-repo.git)
- Approximate number of hosts to be managed?
- Once this information is received, space will be provisioned for your group in Foreman.
Alternatively, you can send a pull request to data/common.yaml to add your tenant configuration. Use the existing records as a guide. Once your pull request is merged, your tenant will be provisioned on the next Puppet run.
- Configure a control repository
- Control repository compatibility adjustments for CLS infrastructure
- An account in CLS Foreman - user.admin privileges required to add, edit hosts
Building your Host
- Infoblox : Get an IP for your machine(s)
- Adding Hosts to Infoblox (video)
- Create Host record in Infoblox (video)
- Register IP in selected VLAN
- Add MAC address
- Configure PXE option (video) :
- "PXE-CLS-Option" for PXE linux installs
- "PXE-CLS-UEFI-Option" for UEFI PXE Linux installs
Configure Hiera data for your workstation
By default, this data is stored in the "data/nodes" directory in your control repository in a file called : hostname.yaml To include this workstation in a group, this data would be set in the appropriate group.yaml file in "data/groups".
Hiera is used to provide configuration data for your workstation. To configure a host at a basic level, add a YAML file to your control repository that is named after the host it configures (i.e. myhost.yaml). There are more complex arrangements that solve duplication of configs and more, but those are documented elsewhere.
This example configuration :
- Joins the machine to Active Directory
- Limits access to a few individuals
- Defines update time
- Defines nightly reboot time
- Shows options for Linux Malware Detect
This configuration can be included in the <hostname>.yaml file (where <hostname> is the FQDN of your host), which is in the data/nodes directory
# hieradata/nodes/workstation.faculty.ncsu.edu.yaml
# this is an example yaml file to define the configuration
# of a workstation. This would be typical for an owner
# with id 'faculty23' and grad students 'cagrad12' and 'jqgrad3'
# directives for higher level classification would go in manifests/site.pp
# Install a terminal-only workstation or server (uncomment next line)
# include role::workstation_micro
# Update the machine at midnight, every day.
profile::base::automatic_updates: "0 0 * * *"
# Reboot nightly at 2:00am.
profile::base::scheduled_reboot::schedule: "0 2 * * 0"
# Authenticate against Active Directory and use
# local home directories (rather than AFS)
profile::auth::provider: ad
profile::auth::mkhomedir_location: local
# Any of these users can get a local session.
profile::auth::allow_users:
- faculty23
- cagrad12
- jqgrad3
# Only `faculty23` can run `sudo`.
profile::auth::sudoers:
- faculty23
# Only `faculty23` can SSH to this machine.
profile::remote_access::ssh_allow_users:
- faculty23
## Linux malware detect : default options
# daily_scan: true
# autoupdate_signatures: '1' (yes)
# autoupdate_version: '0' (no)
# cron_prune_days: '21'
## Configure optional requirements
# - Disk partition layout (part of Foreman configuration)
# - Remote SSH access allowed? (yes,no)
# - Reboot system monthly (yes,no)
# - Update system daily (yes,no)
# - Mount remote network shares? (no, specify shares)
# - AFS support? (yes,no)
# - Use AFS home space or local home space? (AFS, local)
# - Additional software (none, specify list)
# - Printers (none, list of printers) - can this be done as part of the install?
There are more options detailed in the CLS documentation. These should be adequate for most situations
Create a host record in Foreman
Before you proceed, please note :
- <unity_id>.admin account required for CLS Foreman access
- Selecting “Cancel” from any tab will cancel the entire creation process
- Do not select “Submit” until all the tabs have been completed
- Always fill out the screens from the top down. Doing otherwise can alter the inputs.
Select Hosts > Create Host
then process following the sections of the host entry form
Select Host tab
- Name (short name, not fully qualified)
- Organization (defaults to yours)
- Location (use "Default Location)
- Host Group (your organization)
- Deploy on (“Bare metal”)
- Environment (use your default)
- Puppet Proxy (use default "Puppet")
- Puppet CA Proxy (use default "Puppet CA")
Select Operating System tab
- Architecture - Usually “x86_64”
- Operating system
- CentOS 7
- RedHat Enterprise Linux 7 (Server)
- Ubuntu 18.04 LTS
- Ubuntu 20.04 LTS
- Build Mode - checked
- Media - Select media type
- RedHat Server Media
- RedHat Workstation Media
- Ubuntu Mirror
- Partition table - Select “OIT-Linux Supported Partitioning” **????**
- PXE Loader - Select “PXELinux BIOS” (for UEFI install, use GRUB2 UEFI). Note that this is different from the filter setting in Infoblox, which is PXE-CLS-UEFI-OPTION
- Root password - initial root password used for setup
- Provisioning templates - Click the “Resolve” button to insure selected options are compatible
Note : Root password must be longer than 8 characters and relatively complex
Whether or not root is allowed to SSH to your machine is configurable via profile::remote_access::ssh_root_login and is defaulted to no, for Puppet Roles that have that profile applied.
Note: The RedHat default configuration "out of the box" will allow for root logins until puppet runs for the first time to change it
Note : Ubuntu does not allow root logins
Select Interfaces tab
Configure the interface that will be used to PXE boot to our deployment of Foreman
Click “Edit” (or click “Add Interface” if you want to add an additional one)
- Type - Interface
- MAC address : same as in VM **
- Device identifier - leave blank
- DNS name - same short DNS name
- Domain - Same as configured in DNS **
- IPv4 Subnet - select the VLAN as it is defined for your host in DNS **
- IPv6 Subnet - do not change
- IPv4 address - leave blank
- IPv6 address - leave blank
- Managed - checked **
- Primary - checked **
- Provision - checked **
- Virtual NIC - DO NOT CHECK
Select “OK” to close Interface dialog
** : required
Select Puppet Classes tab
Puppet Classes are currently disabled
Select Parameters tab
Assign any host parameters which may influence provisioning There are some parameters you may need to be aware of to set at either the host group or host level, depending on your need.
This will be typical of what you will see :
Select Additional Information tab
- Owned by : your Admin ID
- Hardware Model : Optional
- Comment : Optional
The machine will be provisioned if deployed to anything other than Bare Metal, and networking information will be assigned, if available.
After a host is created, it will automatically enter Build Mode, which can be manually toggled at any time. When a host enters Build Mode, a new provisioning token is generated, which is good for an hour.
Red Hat Enterprise Linux
RedHat operating systems will subscribe to the CDN by default in the CLS foreman. No additional configuration is required at this time.
CLS Provisioning Process
- Client boots, obtains DHCP lease with bootp server info
- Client starts PXE boot process, sending MAC address to Foreman
- Foreman authorizes the client based on the MAC address
- Foreman sends PXE templates depending on boot selection for host
- PXE templates will include links to kernel and initramfs images
- PXE templates will include links to provision templates (kickstart, etc)
- Client executes the templates, which provision the machine
- Provisioning templates are secured with a token, which expires after an hour. Client obtains the token in the initial PXE configs, auth’d via MAC.
PXE booting your host for the first time
After entering the details for your host, click "Submit". You will be redirected to the host interface for your newly created machine and "Build" will have been selected (marking your machine ready for PXE booting). Clicking "Build" tells Foreman to write a TFTP configuration for your host, which is used during the PXE boot process. If you forget to click this, the machine will chain-load to disk, by default.
PXE / UEFI PXE boot the machine. You should eventually see output from the PXE process that shows your machine pulling boot images and starting up. UEFI PXE boot may take a few seconds before any output appears. Diagnosing problems with PXE booting can be tricky. If you ever need help or want to know if your box is actually communicating with our deployment of Foreman, let us know in #cls and we can help. Some good places to start if you have trouble are:
- Verify (from PXE output) that you are getting the correct IP address
- Verify that the MAC address of your hardware matches what is in Infoblox and Foreman
- If you have permissions, search the firewall index in Splunk to see if packets are getting dropped by network firewalls. (include example search)
Waiting on the first Puppet run
The CLS deployment of Foreman "releases" ownership of the host to you after bare-metal provisioning. It is likely running Puppet for the first time and will be made available to you. Log in as root if necessary, but this means that you shouldn't expect the system to be ready for an end-user after the first boot.
Puppet runtime is highly dependent on the type of host being provisioned. A role::workstation_minimal host usually builds in 12-14 minutes after OS has been installed. However a role::workstation can take more than 30-45 minutes to build. This is because a desktop environment is made up of many packages and takes time to install. Likewise, any host that uses our software profile to install many packages will take longer during the first Puppet run. This is usually not a problem, but is something to consider when it comes to support for your end-users.
When the first run completes, you will see two reports on the host interface in Foreman:
- A "plugin-sync" Puppet run executed before the machine was released by Foreman.
- The actual Puppet run that applied configuration.
At this point, the machine is ready for folks to use. Good luck!
- Contact support if build fails, continue until success
- Other processes to be defined
- Request to give another user login or sudo access
- Request to install additional software
Information on specific regulatory builds
EPS compliant workstation
Hosts that connect to the campus network need to be EndPoint Protection Standard (EPS) compliant. This ensures that your system is protected from potential security threats on the Internet.
- For EPS desktop, use role:workstation_micro
- For EPS desktop, add additional profiles to incorporate optional requirements
- Authentication - satisfied with profile::auth and profile::remote_access
- Encrypted Network Communication - satisfied with profile::remote_access
- Host-based Firewall - satisfied with profile::remote_access and profile::firewall
- Least privilege Access - satisfied with profile::auth and profile::remote_access
- Authorization
- Remote Access - satisfied with profile::remote_access
- Software Inventory - satisfied with installed_software Puppet fact
- Network Storage
- Log forwarding
- Supported builds (confirm with IWS)
- RHEL 8
- Ubuntu 18.04 LTS
- Ubuntu 20.04 LTS
- Foreman settings
- Select OS from Operating system tab in Foreman
- any additional information
- Puppet Settings
- Default EPS manifest
- any additional information
ALL the above profiles and settings are included to satisfy specific EPS requirements. For example, the profile::remote_access ensures compliance with EPS Encrypted Network Communications (ENC). If you need to modify or remove any of the profiles or settings during initial build or subsequent support, you are required to first obtain an approval for the exception from Security and Compliance.
In addition to these profiles and settings, EPS compliance also depends on essential security measures of respective OS and software packages. For example, all modern browsers support Web Reputation Filtering (WRF), which is an EPS requirement. You need to obtain an approval from Security and Compliance before disabling such functionality.
At the present time, there is no automated procedure to manage full disk encryption even though it is strongly recommended. You can use the following procedure to enable full disk encryption (on Ubuntu or RedHat systems). Please securely protect your encryption key. You will not be able to recover any data from the disk later if the encryption key is lost.
DIsk Encryption
If you wish to employ full disk encryption, that will need to be configured as part of the install and is beyond the scope of this process. The CLS install process supports full disk encryption during install using a custom kickstart file, but does not escrow the key. This would have to be managed physically and would be required whenever the system booted.
The performance on older hardware may also be a concern as everything on the drive will be encrypted, including swap space and system partitions.
The other option is to encrypt your home directory or specific partitions. This can be done after the fact on both supported systems
RedHat
Disk Encryption via LUKS - for "after the install" this process will delete the contents of your /home partition before recreating the encrypted partition. Note that the data is only protected when the disk is off...once the system is booted, all content on the disk is available
Ubuntu
CLS SPECIFIC CONTENT
Configure a control repository
Each tenant in our deployment of Foreman will have an r10k control repository that specifies module dependencies via Puppetfile as well as node classifications via manifests/*.pp. You should use the
skeleton control repository provided by Puppetlabs. You are free to execute whatever Puppet you want out of this environment.
The traditional usage of a Puppetfile declares every dependency that your Puppet code will need to run. The Infrastructure Puppetfile will always have the full list of modules needed to run any of the shared modules maintained by the CLS. You can simply copy-paste this file into your repository, and add any additional modules you need as you go.
Check out the Example Puppetfiles for a detailed example.
Once you've pushed your repository to GitHub, grant the cls-puppet-svc GitHub user read. This allows automation in our environment to clone your control repository when changes occur.
If you wish to set up a "push-to-deploy" integration between GitHub Enterprise and Foreman, check out Using GitHub Webhooks to Automatically Deploy via r10k.
Control repository compatibility adjustments for CLS infrastructure
If you wish to make use of the shared configurations provided by CLS, one of the things you will need to do first is clean up the control repository provided by PuppetLabs a bit. The following steps should be taken:
- Delete or rename site-modules/role and site-modules/profile as it conflicts with the CLS provided roles. It is recommended that you use something like {tenant_slug}_role and {tenant_slug}_profile for your own roles and profiles.
- If you're just starting out, remove the hierarchy configuration in your hiera.yaml, and add data_hash: yaml_data to defaults. Your hiera.yaml should look like the following at this point.
Source - this is a copy of content generated from https://github.ncsu.edu/twk/infrastructure/blob/patch-1/docs/getting-started.md#getting-started