Learn how to install Chef on Ubuntu…

Chef is a configuration management tool based on ruby. It is used to automate the management of configurations across all the nodes in the cluster and maintain consistency. There are three major components of Chef.

  • Chef server
  • Workstation
  • Nodes

All the configurations are managed from workstations and later pushed on to the chef-server. Chef server is the centralized store of all the infrastructure configurations. Knife is a command-line tool present on a workstation that is used to interact with Chef Server. Nodes run chef clients who ask for configuration information from the chef server.

This is how a typical Chef architecture looks like:

chef architecture

Now let’s go ahead and setup chef Server, Workstation, and Node (Chef Client).

Environment Details

I am using 3 Ubuntu 18.04 systems. One will act as a chef server, and the next one will be a workstation, and a 3rd system will be the node.

Chef Server

  • Hostname: chef-geekflare
  • IP Address: 192.168.0.107

Workstation

  • Hostname: workstation
  • IP Address: 192.168.0.108

Node

  • Hostname: client-node
  • IP Address: 192.168.0.109

Before I begin the installation, I need to edit the /etc/hosts file of all the systems so that they can resolve each other. Edit the /etc/hosts file on all the systems, as shown below.

[email protected]:~$ 
127.0.0.1 localhost 
127.0.1.1 geekflare 
192.168.0.107 chef-geekflare 
192.168.0.108 chef-workstation 
192.168.0.109 client-node

I will run below command on all the three systems to update them.

[email protected]:~$ sudo apt update

[sudo] password for geekflare:

Hit:1 http://security.ubuntu.com/ubuntu cosmic-security InRelease

Get:2 https://download.docker.com/linux/ubuntu bionic InRelease [64.4 kB]     

Hit:3 http://ppa.launchpad.net/ansible/ansible/ubuntu cosmic InRelease        

Hit:4 http://us.archive.ubuntu.com/ubuntu cosmic InRelease                    

Get:5 https://download.docker.com/linux/ubuntu bionic/stable amd64 Packages [9,594 B]

Hit:6 http://us.archive.ubuntu.com/ubuntu cosmic-updates InRelease            

Hit:7 http://us.archive.ubuntu.com/ubuntu cosmic-backports InRelease          

Get:8 http://apt.puppetlabs.com bionic InRelease [85.3 kB]                    

Get:9 http://apt.puppetlabs.com bionic/puppet6 amd64 Packages [32.4 kB]

Fetched 192 kB in 2s (84.6 kB/s)   

Reading package lists... Done

Building dependency tree      

Reading state information... Done

233 packages can be upgraded. Run 'apt list --upgradable' to see them.

Chef Server Installation

Chef Server is a component in the architecture that connects Workstation and Nodes. When the configurations are edited/changed on a workstation, they are pushed to the chef-server, and all the nodes pull these configuration changes from the Chef Server.

Now, let us run the command below to download the chef-server package.

[email protected]:~$ wget https://packages.chef.io/files/stable/chef-server/13.0.17/ubuntu/18.04/chef-server-core_13.0.17-1_amd64.deb

--2019-10-23 04:04:35-- https://packages.chef.io/files/stable/chef-server/13.0.17/ubuntu/18.04/chef-server-core_13.0.17-1_amd64.deb

Saving to: ‘chef-server-core_13.0.17-1_amd64.deb’

chef-server-core_13 100%[===================>] 240.58M 1.33MB/s in 6m 16s 

2019-10-23 04:10:51 (656 KB/s) - ‘chef-server-core_13.0.17-1_amd64.deb’ saved [252269838/252269838]

Now you need to run the following command install the chef server.

[email protected]:~$ sudo dpkg -i chef-server-core_*.deb

chef-server-ctl is command-line utility in chef-server. I will use this utility to start the chef-server services.

[email protected]:~$ sudo chef-server-ctl reconfigure

Running handlers:

Running handlers complete

Chef Infra Client finished, 481/1028 resources updated in 04 minutes 08 seconds

Chef Server Reconfigured!

You can check the status of the services which have started using the below command.

[email protected]:~$ sudo chef-server-ctl status

run: bookshelf: (pid 2452) 822s; run: log: (pid 29553) 951s

run: nginx: (pid 2318) 826s; run: log: (pid 30216) 908s

run: oc_bifrost: (pid 2296) 827s; run: log: (pid 29240) 996s

run: oc_id: (pid 2304) 826s; run: log: (pid 29308) 979s

run: opscode-erchef: (pid 2511) 822s; run: log: (pid 29707) 946s

run: opscode-expander: (pid 2416) 822s; run: log: (pid 29412) 958s

run: opscode-solr4: (pid 2393) 824s; run: log: (pid 29358) 964s

run: postgresql: (pid 2264) 827s; run: log: (pid 28769) 1021s

run: rabbitmq: (pid 3183) 792s; run: log: (pid 30476) 902s

run: redis_lb: (pid 30011) 926s; run: log: (pid 30010) 926s

Create User and Organization

Chef server connects workstation and client nodes. To link them, I will create an admin and organizer with their private keys.

 Firstly, create a .chef directory to store the keys.

[email protected]:~$ mkdir .chef

Now, I will use chef-server-ctl to create a user. In the command below, chefadmin is the user, Chef is the first name, GeekFlare is the last name, [email protected] is the email id, geekflare is the password, chefadmin.pen is the RSA key.

[email protected]:~$ sudo chef-server-ctl user-create chefadmin Chef GeekFlare [email protected] 'geekflare' --filename ~/.chef/chefadmin.pem

Let us run a command to check the list of users on the chef server.

[email protected]:~$ sudo chef-server-ctl user-list

chefadmin

pivotal

Now, I will use chef-server-ctl to create an organization. In the command below, chef-org is the organization name, Geekflare Chef Infrastructure is the full org name, chefadmin is the user we just created., chef-org.pem is the RSA key.

[email protected]:~$ sudo chef-server-ctl org-create chef-org "Geekflare Chef Infrastructure" --association_user chefadmin --filename ~/.chef/chef-org.pem

 Let us run a command to check the list of organizations on the chef server.

[email protected]:~$ sudo chef-server-ctl org-list

chef-org

I have installed the chef-server completely, let us go ahead and install workstation where all the configurations are created.

Workstation

Workstation is the place where users create cookbooks. Cookbooks are nothing but the configuration units which are created to run specific tasks.

Let us run the command below to download the chef workstation package.

[email protected]:~$ wget https://packages.chef.io/files/stable/chef-workstation/0.2.43/ubuntu/18.04/chef-workstation_0.2.43-1_amd64.deb

--2019-10-23 05:37:41-- https://packages.chef.io/files/stable/chef-workstation/0.2.43/ubuntu/18.04/chef-workstation_0.2.43-1_amd64.deb

Resolving packages.chef.io (packages.chef.io)... 151.101.194.110, 151.101.130.110, 151.101.66.110, ...

Connecting to packages.chef.io (packages.chef.io)|151.101.194.110|:443... connected.

HTTP request sent, awaiting response... 200 OK

Length: 129713682 (124M) [application/x-debian-package]

Saving to: ‘chef-workstation_0.2.43-1_amd64.deb’

chef-workstation_0.2.43-1_ 100%[=======================================>] 123.70M 2.37MB/s in 4m 25s 

2019-10-23 05:42:18 (477 KB/s) - ‘chef-workstation_0.2.43-1_amd64.deb’ saved [129713682/129713682]

Let us run dpkg command to install a workstation on the ubuntu system.

[email protected]:~$ sudo dpkg -i chef-workstation_*.deb

Selecting previously unselected package chef-workstation.

(Reading database ... 273360 files and directories currently installed.)

Preparing to unpack chef-workstation_0.2.43-1_amd64.deb ...

Unpacking chef-workstation (0.2.43-1) ...

Setting up chef-workstation (0.2.43-1) ...

To run the experimental Chef Workstation App, use your

platform's package manager to install these dependencies:

               libgconf-2.so.4 => not found

You can then launch the App by running 'chef-workstation-app'.

The App will then be available in the system tray.

Thank you for installing Chef Workstation!

You can find some tips on getting started at https://chef.sh/

I will now run a command to create a chef repository that will have all the cookbooks and other files.

[email protected]:~$ chef generate repo chef-repo

Recipe: code_generator::repo

 * directory[/home/geekflare/chef-repo] action create

   - create new directory /home/geekflare/chef-repo

 * template[/home/geekflare/chef-repo/LICENSE] action create_if_missing

   - create new file /home/geekflare/chef-repo/LICENSE

   - update content in file /home/geekflare/chef-repo/LICENSE from none to 3c525c

   (diff output suppressed by config)

 * cookbook_file[/home/geekflare/chef-repo/.chef-repo.txt] action create_if_missing

   - create new file /home/geekflare/chef-repo/.chef-repo.txt

   - update content in file /home/geekflare/chef-repo/.chef-repo.txt from none to 2bed28

   (diff output suppressed by config)

 * cookbook_file[/home/geekflare/chef-repo/README.md] action create_if_missing

   - create new file /home/geekflare/chef-repo/README.md

   - update content in file /home/geekflare/chef-repo/README.md from none to 2b4f46

   (diff output suppressed by config)

 * cookbook_file[/home/geekflare/chef-repo/chefignore] action create_if_missing

   - create new file /home/geekflare/chef-repo/chefignore

   - update content in file /home/geekflare/chef-repo/chefignore from none to 9e2ffd

   (diff output suppressed by config)

 * remote_directory[/home/geekflare/chef-repo/cookbooks] action create_if_missing

   - create new directory /home/geekflare/chef-repo/cookbooks 

Recipe: code_generator::repo

 * cookbook_file[/home/geekflare/chef-repo/cookbooks/README.md] action create_if_missing

   - create new file /home/geekflare/chef-repo/cookbooks/README.md

   - update content in file /home/geekflare/chef-repo/cookbooks/README.md from none to 54b03d

   (diff output suppressed by config)

 * execute[initialize-git] action run

   - execute git init .

 * template[/home/geekflare/chef-repo/.gitignore] action create_if_missing

   - create new file /home/geekflare/chef-repo/.gitignore

   - update content in file /home/geekflare/chef-repo/.gitignore from none to 11e5ee

   (diff output suppressed by config)

Now I will create /chef-repo/.chef directory, which will store all the knife configurations and the RSA keys.

[email protected]:~$ mkdir ~/chef-repo/.chef

[email protected]:~$ cd chef-repo/ 

Now let us generate RSA key-pair. We are generating this key to authenticate the workstation and get access to the chef server.

[email protected]:~/chef-repo$ ssh-keygen -b 4096

Generating public/private RSA key pair.

Enter file in which to save the key (/home/geekflare/.ssh/id_rsa):

Enter passphrase (empty for no passphrase):

Enter same passphrase again:

Your identification has been saved in /home/geekflare/.ssh/id_rsa.

Your public key has been saved in /home/geekflare/.ssh/id_rsa.pub.

The key fingerprint is:

SHA256:oK/ZyLn+AOMj97F5Z0e1K5o1bxChyKx3ms4HvK06DxI [email protected]

The key's randomart image is:

+---[RSA 4096]----+

| |

| . |

| o.. . . |

| .+.. . . |

| E .o S o . |

| . +..+ . o . |

|. = +..B .o. . |

| o =.&= =oooo |

| .&OB=oo o. |

+----[SHA256]-----+

Now copy the key to the chef-server from the workstation.

[email protected]:~/chef-repo$ sudo ssh-copy-id [email protected]

/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/home/geekflare/.ssh/id_rsa.pub"

/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed

/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys

[email protected]'s password:

Number of key(s) added: 1

Now try logging into the machine, with: "ssh '[email protected]'"

and check to make sure that only the key(s) you wanted were added.

Now I will copy the .pem files (chefadmin.pem and chef-org.pem) from the chef server to the workstation.

[email protected]:~/chef-repo$ scp [email protected]:~/.chef/*.pem ~/chef-repo/.chef/

chefadmin.pem 100% 1674 105.1KB/s 00:00   

chef-org.pem 100% 1674 103.0KB/s 00:00

Check if .pem files got copied successfully on the workstation.

[email protected]:~/chef-repo$ ls ~/chef-repo/.chef

chefadmin.pem chef-org.pem

Create Version Control

While working on a workstation, a lot of changes and editing happens in cookbooks (configuration units), so a version control system is required to keep track of these changes. So, let us go ahead and create a version control system using Git on the workstation. I will create a git repository inside the chef-repo directory. I will add a username and email to configure git

[email protected]:~$ git config --global user.name geekflare

[email protected]:~$ git config --global user.email [email protected]

Let us add the .chef directory to the .gitignore file.

[email protected]:~$ echo ".chef" > ~/chef-repo/.gitignore

Now I will run add and commit git commands from a chef-repo directory.

[email protected]:~$ cd ~/chef-repo

[email protected]:~/chef-repo$ git add .

[email protected]:~/chef-repo$ git commit -m "initial commit"

[master (root-commit) 99c8c11] initial commit

 16 files changed, 359 insertions(+)

 create mode 100644 .chef-repo.txt

 create mode 100644 .gitignore

 create mode 100644 LICENSE

 create mode 100644 README.md

 create mode 100644 chefignore

 create mode 100644 cookbooks/README.md

 create mode 100644 cookbooks/example/README.md

 create mode 100644 cookbooks/example/attributes/default.rb

 create mode 100644 cookbooks/example/metadata.rb

 create mode 100644 cookbooks/example/recipes/default.rb

 create mode 100644 data_bags/README.md

 create mode 100644 data_bags/example/example_item.json

 create mode 100644 environments/README.md

 create mode 100644 environments/example.json

 create mode 100644 roles/README.md

 create mode 100644 roles/example.json

Check the status.

[email protected]:~/chef-repo$ git status

On branch master

nothing to commit, working tree clean

Generate Your First CookBook

Now the installation of WorkStation is complete, and you can start creating cookbooks on the WorkStation. Try generating a sample cookbook on the WorkStation and see if it gets generated successfully.

I will run the below command to generate a cookbook.

[email protected]:~$ Chef generate cookbook geekflare_cookbook

Generating cookbook geekflare_cookbook

- Ensuring correct cookbook file content

- Committing cookbook files to Git

- Ensuring delivery configuration

- Ensuring correct delivery build cookbook content

- Adding delivery configuration to feature branch

- Adding build cookbook to feature branch

- Merging delivery content feature branch to master

Your cookbook is ready. Type `cd geekflare_cookbook` to enter it.

There are several commands you can run to get started locally developing and testing your cookbook.

Type `delivery local --help` to see a full list.

Why not start by writing a test? Tests for the default recipe are stored at:

test/integration/default/default_test.rb

If you'd prefer to dive right in, the default recipe can be found at:

recipes/default.rb

Generate chef-repo, then move to a chef-repo directory

[email protected]:~$ Chef generate app chef-repo

WARNING: The command 'chef generator app' is deprecated and will be removed from the next major release of Chef DK / Workstation (April 2019)

Recipe: code_generator::app

 * directory[/home/geekflare/chef-repo] action create (up to date)

 * template[/home/geekflare/chef-repo/.kitchen.yml] action create

   - create new file /home/geekflare/chef-repo/.kitchen.yml

   - update content in file /home/geekflare/chef-repo/.kitchen.yml from none to ceae09

   (diff output suppressed by config)

 * directory[/home/geekflare/chef-repo/test/integration/default] action create

   - create new directory /home/geekflare/chef-repo/test/integration/default

 * template[/home/geekflare/chef-repo/test/integration/default/default_test.rb] action create_if_missing

   - create new file /home/geekflare/chef-repo/test/integration/default/default_test.rb

   - update content in file /home/geekflare/chef-repo/test/integration/default/default_test.rb from none to 0f757b

   (diff output suppressed by config)

 * template[/home/geekflare/chef-repo/README.md] action create

   - update content in file /home/geekflare/chef-repo/README.md from 2b4f46 to 6401b8

   (diff output suppressed by config)

 * directory[/home/geekflare/chef-repo/cookbooks] action create (up to date)

 * directory[/home/geekflare/chef-repo/cookbooks/chef-repo] action create

   - create new directory /home/geekflare/chef-repo/cookbooks/chef-repo

 * template[/home/geekflare/chef-repo/cookbooks/chef-repo/metadata.rb] action create

   - create new file /home/geekflare/chef-repo/cookbooks/chef-repo/metadata.rb

   - update content in file /home/geekflare/chef-repo/cookbooks/chef-repo/metadata.rb from none to e30be3

   (diff output suppressed by config)

 * cookbook_file[/home/geekflare/chef-repo/cookbooks/chef-repo/chefignore] action create

   - create new file /home/geekflare/chef-repo/cookbooks/chef-repo/chefignore

   - update content in file /home/geekflare/chef-repo/cookbooks/chef-repo/chefignore from none to 9e2ffd

   (diff output suppressed by config)

 * cookbook_file[/home/geekflare/chef-repo/cookbooks/chef-repo/Berksfile] action create

   - create new file /home/geekflare/chef-repo/cookbooks/chef-repo/Berksfile

   - update content in file /home/geekflare/chef-repo/cookbooks/chef-repo/Berksfile from none to 15e000

   (diff output suppressed by config)

 * directory[/home/geekflare/chef-repo/cookbooks/chef-repo/recipes] action create

   - create new directory /home/geekflare/chef-repo/cookbooks/chef-repo/recipes

 * template[/home/geekflare/chef-repo/cookbooks/chef-repo/recipes/default.rb] action create

   - create new file /home/geekflare/chef-repo/cookbooks/chef-repo/recipes/default.rb

   - update content in file /home/geekflare/chef-repo/cookbooks/chef-repo/recipes/default.rb from none to f56ecb

   (diff output suppressed by config)

 * directory[/home/geekflare/chef-repo/cookbooks/chef-repo/spec/unit/recipes] action create

   - create new directory /home/geekflare/chef-repo/cookbooks/chef-repo/spec/unit/recipes

 * cookbook_file[/home/geekflare/chef-repo/cookbooks/chef-repo/spec/spec_helper.rb] action create_if_missing

   - create new file /home/geekflare/chef-repo/cookbooks/chef-repo/spec/spec_helper.rb

   - update content in file /home/geekflare/chef-repo/cookbooks/chef-repo/spec/spec_helper.rb from none to 1f80e1

   (diff output suppressed by config)

 * template[/home/geekflare/chef-repo/cookbooks/chef-repo/spec/unit/recipes/default_spec.rb] action create_if_missing

   - create new file /home/geekflare/chef-repo/cookbooks/chef-repo/spec/unit/recipes/default_spec.rb

   - update content in file /home/geekflare/chef-repo/cookbooks/chef-repo/spec/unit/recipes/default_spec.rb from none to 666a01

   (diff output suppressed by config)

 * execute[initialize-git] action run

   - execute git init .

 * cookbook_file[/home/geekflare/chef-repo/.gitignore] action create

   - update content in file /home/geekflare/chef-repo/.gitignore from 25558e to edcd62

   (diff output suppressed by config)

Configure Knife

Knife is a command-line tool to manage nodes, cookbooks, and recipes. To configure Knife, create a config.rb file and put the below content in the file, these are knife configurations.

[email protected]:~$ sudo gedit ~/chef-repo/.chef/config.rb
current_dir = File.dirname(__FILE__)

log_level :info

log_location STDOUT

node_name 'chefadmin'

client_key "chefadmin.pem"

validation_client_name 'chef-org-validator'

validation_key "chef-org-validator.pem"

chef_server_url 'https://chef-geekflare/organizations/chef-org'

cache_type 'BasicFile'

cache_options( :path => "#{ENV['HOME']}/.chef/checksums" )

cookbook_path ["#{current_dir}/../cookbooks"]

Now go to a chef-repo directory and copy SSL certificates.

[email protected]:~$ cd chef-repo

[email protected]:~/chef-repo$ knife ssl fetch

WARNING: Certificates from chef-geekflare will be fetched and placed in your trusted_cert

directory (/home/geekflare/chef-repo/.chef/trusted_certs).

Knife has no means to verify these are the correct certificates. You should

verify the authenticity of these certificates after downloading.

Adding certificate for chef-geekflare in /home/geekflare/chef-repo/.chef/trusted_certs/chef-geekflare.crt

To check that config.rb is set correctly, run the command below.

[email protected]:~/chef-repo$ knife client list

chef-org-validator

Bootstrap a Node

Bootstrap runs from the workstation machine and installs the chef-client on the nodes. The nodes can then read configurations from the chef server by using the client node’s user and password to bootstrap a node.

I will now bootstrap a node with IP address 192.168.0.109, user name geekflare, and password geekflare.org.

[email protected]:~/chef-repo/.chef$ knife bootstrap 192.168.0.109 -x geekflare -P geekflare.org --node-name geekflare-client-1

Creating new client for geekflare-client-1

Creating new node for geekflare-client-1

Connecting to 192.168.0.109

192.168.0.109 -----> Installing Chef Omnibus (-v 14)

192.168.0.109 downloading https://omnitruck-direct.chef.io/chef/install.sh

192.168.0.109 to file /tmp/install.sh.9250/install.sh

192.168.0.109 trying wget...

192.168.0.109 ubuntu 18.10 x86_64

192.168.0.109 Getting information for chef stable 14 for ubuntu...

192.168.0.109 downloading https://omnitruck-direct.chef.io/stable/chef/metadata?v=14&p=ubuntu&pv=18.10&m=x86_64

192.168.0.109 to file /tmp/install.sh.9261/metadata.txt

192.168.0.109 trying wget...

192.168.0.109 sha1 534bae390bde3bd9d93bef99335f62246624f32b

192.168.0.109 sha256 94bc60b3a97ddadf77a70c7678ec77a676942c74f8152a2c70a0f5b68e22a42e

192.168.0.109 url https://packages.chef.io/files/stable/chef/14.14.25/ubuntu/18.04/chef_14.14.25-1_amd64.deb

192.168.0.109 version 14.14.25

192.168.0.109 downloaded metadata file looks valid...

192.168.0.109 downloading https://packages.chef.io/files/stable/chef/14.14.25/ubuntu/18.04/chef_14.14.25-1_amd64.deb

192.168.0.109 to file /tmp/install.sh.9261/chef_14.14.25-1_amd64.deb

192.168.0.109 trying wget...

192.168.0.109 Comparing checksum with sha256sum...

192.168.0.109 Installing chef 14

192.168.0.109 installing with dpkg...

192.168.0.109 Selecting previously unselected package chef.

(Reading database ... 204803 files and directories currently installed.)

192.168.0.109 Preparing to unpack .../chef_14.14.25-1_amd64.deb ...

192.168.0.109 Unpacking chef (14.14.25-1) ...

192.168.0.109 Setting up chef (14.14.25-1) ...

192.168.0.109 Thank you for installing Chef Infra Client! For help getting started visit https://learn.chef.io

192.168.0.109 Starting the first Chef Client run...

192.168.0.109 Starting Chef Client, version 14.14.25

192.168.0.109 resolving cookbooks for run list: []

192.168.0.109 Synchronizing Cookbooks:

192.168.0.109 Installing Cookbook Gems:

192.168.0.109 Compiling Cookbooks...

192.168.0.109 [2019-10-23T10:52:57-04:00] WARN: Node geekflare-client-1 has an empty run list.

192.168.0.109 Converging 0 resources

192.168.0.109

192.168.0.109 Running handlers:

192.168.0.109 Running handlers complete

192.168.0.109 Chef Client finished, 0/0 resources updated in 07 seconds

I will now list all the nodes which got bootstrapped

[email protected]:~/chef-repo/.chef$ knife node list

geekflare-client-1

Run below command to get the details of the node.

[email protected]:~/chef-repo/.chef$ knife node show geekflare-client-1

Node Name: geekflare-client-1

Environment: _default

FQDN: client-node

IP: 192.168.0.109

Run List:   

Roles:      

Recipes:    

Platform: ubuntu 18.10

Tags:

Now the setup is ready!

We have successfully installed a chef server, workstation, and a node on Ubuntu. You can go ahead and start creating recipes and cookbooks in Chef for configuration management of infrastructure.

If you are an absolute beginner, then you may like taking this Udemy course too.