Dotfiles: Setup any new machine in minutes

Automate Mac Mini Setup

Automating Mac Mini (with M1 chip) system configuration

Today I received my new Mac Mini (with M1chip), YAY!! Last few months I had a terrible experience with my 10-year-old Mini. It was saying how many years do you use me, upgrade man. Every few days it restarts, the browser crashes and all in-built apps do not even open. Sometimes, the only option I get is to reinstall the operating system and start from scratch to survive for a couple more weeks. So finally, I decided to upgrade my computer and present myself with a new Mac Mini.

Where to start...

There are many articles related to automating system configuration from scratch and well-known dotfiles repositories to start with. Some were created years back and needed a few updates per new operating system requirements. This article shows my way of setting up and I can use it as a reference in case I need it in the future.

Dotfiles:

Dotfiles are commonly used for storing user preferences or preserving the state of a utility/application and are frequently created implicitly by using various utilities. So far, I kept all my configuration files and some scripts in one of the cloud drives and maintained notes to set up new instance of my mini. This time, I would like to automate the configuration process while setting up my new machine so that I don’t need to manually download the folder from the cloud and update the configuration files. Here are the steps that I followed to configure my machine from scratch. As part of the automation, I also decided to save all my configurations in the GitHub dotfiles repository.

First Step (and Last Step)

After finishing the initial first-time boot configuration, open Terminal and install Homebrew which is the package manager to install all other packages. Make sure you have Xcode installed before installing homebrew. You might see the error: “xcrun: error: invalid active developer path (/Library/Developer/CommandLineTools), missing xcrun at: /Library/Developer /CommandLineTools/usr/bin/xcrun”

This means Xcode not installed/not installed properly. Run the following to remove incorrectly installed Xcode:

sudo rm -rf /Library/Developer/CommandLineTools

Download Xcode Command Line Tools either by xcode-select --install or directly from https://developer.apple.com/download/all/?q=xcode. (Select Command Line Tools for Xcode).

Then run below commands to install homebrew.

# Install Homebrew
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

After installing homebrew, there is a warning: /opt/homebrew/bin is not in your PATH and fixed by adding it to my PATH as below.

# Add brew to system Path:
echo 'eval "$(/opt/homebrew/bin/brew shellenv)"' >> ~/.zshrc
eval "$(/opt/homebrew/bin/brew shellenv)"

Also, run the below command if you don’t want your usage details not to be sent outside your network.

# Turn off Analytics (optional):
brew analytics off

As I already created the dotfiles repository in my GitHub and moved all my configuration/setup files from cloud drive to GitHub, now we can clone this repository and finish the configuration of my new machine. Alternatively, we can also run only the `setup.sh` scripts in specific subfolders if we don’t need everything. We can do this as:

# Clone Dotfiles repo for GitHub
git clone https://github.com/nsanka/dotfiles.git
cd dotfiles

# Make all .sh files as executables:
find . -name \*.sh -exec chmod 744 {} \;

We need to update few files before running the complete setup script.

• Update or Remove all the lines that has computer name (MyMiniM1) in macos/setup.sh file.
• Update your name and email in git/.gitconfig file. If you forget you can also update by typing below commands in Terminal later:
• git config — global user.email useremail@example.com. Make sure your email is same as your Databricks email (case-sensitive)
• git config — global user.name “FirstName LastName”
• Check the list of applications those will be installed in packages/Brewfile
• Check the lines in between conda initializein shell/.zshrc file, the conda path maybe different in your machine.
• Check the packages/conda_env.yml file for the list of conda packages for Python Environment.

# Run Complete Setup
./bootstrap.sh

# Setup Only Visual Studio Code (for example)
./vscode/setup.sh

The bootstrap script as shown below first calls setup.sh file in the packages folder and then it just calls all the setup files from all individual folders.

#! /usr/bin/env sh

DIR=$(dirname "$0")
cd "$DIR"

# Activate Some common functions
. scripts/functions.sh

info "Prompting for sudo password..."
if sudo -v; then
   # Keep-alive: update existing `sudo` time stamp
   while true; 
      do sudo -n true; sleep 60; kill -0 "$$" || exit; 
   done 2>/dev/null &
   success "Sudo credentials updated."
else
   error "Failed to obtain sudo credentials."
fi

# Install Packages/Applications first in order for the rest to work
./packages/setup.sh

# Configure system settings and all other preferences
find * -name "setup.sh" -not -wholename "packages*" | while read setup; do
   ./$setup
done

success "Finished configuring Dotfiles"

That’s all, my new machine is ready in a couple of minutes. Now with dotfiles saved in GitHub, I can mess around with any new packages and my system settings quite easily, without ever having to worry about really breaking something important. If I would break something, my dotfiles are always right there to back me up.

Individual Components

As I started with a brand new macOS installation, running the bootstrap script file will perform the following tasks:

• first, install the required packages/applications.
• setup integrated development environment (IDE).
• setup macOS system-wide settings.
• configure shell settings.
• setup some commonly used applications like GitHub, Vim, etc.

It can be a separate article if I have to explain configuring each topic in full detail, but I want to talk about a few of these topics briefly here.

Install Packages/Applications

The packages/setup.sh file installs the necessary applications listed in Brewfile and then it runs the commands listed in *.list files. The Brewfile has the applications which I use regularly, some are brew and some are brew cask applications. Homebrew typically deals with command line software and Cask offers a way to command line manage the installation of graphical applications. You can check all the applications supported here: Brew Formulae and Cask Formulae.

After installing the applications, the setup file runs the commands by grabbing each line from the *.list file. For example, conda.list file has first line “update -y -n base -c defaults conda”, so the command that will run is:

# Update Conda base environment
conda update -y -n base -c defaults conda

Similarly, code \--install-extension.list file has a line “ms-python.python” which will run the below command:

# Install Python extension in VS Code
code --install-extension ms-python.python

Setup Development Environment (IDE)

Visual Studio Code is my favorite Integrated Developer Environment (IDE) because of its clean design, large selection of plugins, and cross-platform support. The Brewfile has it as a cask package and it’s already installed along with all other required packages. We can automate installing all the Plugins for VSCode using the code --install-extension.list file in packages. The sample file is shown below, you can add any plugin to this file and run the setup.sh in the packages folder to add additional plugins.

github.codespaces
ms-azuretools.vscode-docker
ms-python.python
ms-vscode-remote.remote-containers
ms-vscode-remote.remote-ssh

If you would like to install and setup separately follow this, download VS Code for Mac from official site. As I am using Mac with Apple Silicon chip, download the corresponding VS Code and move the “Visual Studio Code.app” to Applications folder. Launch VS Code, we have a nice, clean editor for our development.

If you prefer not to share all your data usage and errors to Microsoft, open settings Code > Preferences > Settings, search for telemetry, and select off from dropdown for Telemetry: Telemetry Level setting.

When you open VS Code, you will see the Welcome Tab. Click on Install Support for Python in Customize section, this will install Python extension for VS Code.

When you open new terminal (Terminal > New Terminal), we can see the terminal (in Mac it is zsh) depending upon your system default. In order to launch VS Code with code command from Terminal,

• Open the command pallette with Command + Shift + P (or F1)
• Type Shell in command palette
• Select Shell Command: Install code in PATH from suggested list.

Python Environment

The Mac comes with inbuilt Python. But I suggest installing Miniconda or Anaconda Individual Edition instead of using the inbuilt Python. Miniconda is a free, easy-to-install package manager, environment manager, and Python distribution with a collection of 1,500+ open-source packages with free community support. It is platform-agnostic, so you can use it whether you are on Windows, macOS, or Linux.

The Brewfile has Miniconda as a cask package and it’s already installed. Now all we need to do is create my default conda environment and install all the conda packages. The approach is similar to installing VSCode extensions as shown above, I created another “conda.list” file in the packages folder and it is called by the setup.sh file. The sample file is shown below.

# Update Conda Environment
update -y -n base -c defaults conda
# Create New Environment
create -y --name python38 python=3.8
# Install Conda Packages
install -y -n python38 pandas

There is also another approach which is creating a conda environment using “conda_env.yml”. For this to work, use “conda.list” file as shown below. This will exactly match all the versions and dependencies of conda packages. The advantage of this approach is that it will install conda packages and also packages installed using pip.

# Update Conda Environment
update -y -n base -c defaults conda
# Create New Environment Using "yml" File
env create -f conda_env.yml

This creates a new conda environment which can be activated with command: conda activate env_name.

Setup macOS system-wide settings

The macos/setup.sh file has various system-wide settings which I prefer to have in Mac Mini. For example, I would like to have my Dock on the left side instead of default bottom. This will be set using the below command:

# Set the orientation of Dock
defaults write com.apple.dock orientation -string "left"

Configure shell settings

The setup.sh file in shell folder has a command which installs Oh-My-Zsh which is an awesome configuration framework for zsh. The Oh-My Zsh will be replaced the default terminal with zsh automatically when you installed. Another interesting feature of Oh-My-Zsh is the endless amount of shell plugins, keyboard extensions and other functionality.

Setup GitHub working with SSH

With SSH keys, we can connect to GitHub without supplying our username or password at each visit.

• Open Terminal
• Paste the text below, substituting in your GitHub Enterprise Server email address.
• ssh-keygen -t ed25519 -C "your_email@example.com"
• Select default values for the rest of the options.
• Start ssh-agent in the background: eval "$(ssh-agent -s)"
• Add below lines to ~/.ssh/config file:

Host github.com
  AddKeysToAgent yes
  UseKeychain yes
  IdentityFile ~/.ssh/id_ed25519

• Add your SSH private key to the ssh-agent as below
• ssh-add ~/.ssh/id_ed25519
• Copy the SSH public key to your clipboard
• pbcopy < ~/.ssh/id_ed25519.pub
• Login to your GitHub account and click your profile photo, then click Settings
• In the user settings sidebar, click SSH and GPG keys
• Click New SSH key or Add SSH key
• Add a descriptive label for the new key in the Title
• Paste your key into the “Key” field
• Add SSH Key
• To test the SSH connection, enter below command in Git Bash and enter yes
• ssh -T git@github.com
• You will see a successful message as below

> Hi username! You've successfully authenticated, but GitHub does not
> provide shell access.

If you see any permission error or want to know more details, check this link.

Docker Setup

The Brewfile has it as cask package and it’s already installed when you run complete setup. If you didn’t install, you can install Docker using brew install --cask docker command. You can verify the installation using docker --version command. Whenever Docker is running, you should also see its icon in the status bar.

Next click the Docker icon in the status bar and go to Preferences, you’ll be able to choose whether Docker should start automatically with system startup. I suggest starting automatically with system.

As I have Mac Mini with Apply M1 chip, all the docker images that we pull will be of arm64 architecture. Docker images built with Apple Silicon (or another ARM64 based architecture) can create issues when deploying the images to a Linux or Windows based *AMD64 environment (e.g. AWS EC2, ECS, etc.). For example, you may try to upload your docker image made on the M1 chip to an AWS ECR repository and it fails to run. Therefore, you need a way to build AMD64 based images on the ARM64 architecture, whether it’s using Docker build (for individual images) or docker-compose build (e.g. for multi-image apps running in a docker compose network).

For building single docker images: Set your environment variable using the command line or modifying your .bashrc or .zshrc file. It’s already there in .zshrc file that you downloaded from git repository, please confirm if it's there or not.

export DOCKER_DEFAULT_PLATFORM=linux/amd64

To use the Docker support in VS Code, we need to install the VS Code Docker Extension. In VS Code, Open the Extensions tab (Ctrl+Shift+X) and search for Docker. Select the extension published by Microsoft and click install. It will add a Docker icon to the Activity Bar.

Conclusions

My Mac Mini is ready for use with all my preferred settings and applications. All my settings/preferences are saved in GitHub dotfiles repository, so that I can quickly configure any other Mac in minutes.

If you enjoy reading my articles and want to support me, please consider signing up to become a Medium member. It’s $5 a month and gives you unlimited access to stories on Medium. Please signup using my link to support me: https://nsanka.medium.com/membership.