Learn how to install Chef on Ubuntu…
Chef is a configuration management tool based on ruby. It is used to automate the management of configurations across all the nodes in the cluster and maintain consistency. There are three major components of Chef.
- Chef server
- Workstation
- Nodes
All the configurations are managed from workstations and later pushed on to the chef-server. Chef server is the centralized store of all the infrastructure configurations. Knife is a command-line tool present on a workstation that is used to interact with Chef Server. Nodes run chef clients who ask for configuration information from the chef server.
This is how a typical Chef architecture looks like:
Now let’s go ahead and setup chef Server, Workstation, and Node (Chef Client).
Environment Details
I am using 3 Ubuntu 18.04 systems. One will act as a chef server, and the next one will be a workstation, and a 3rd system will be the node.
Chef Server
- Hostname: chef-geekflare
- IP Address: 192.168.0.107
Workstation
- Hostname: workstation
- IP Address: 192.168.0.108
Node
- Hostname: client-node
- IP Address: 192.168.0.109
Before I begin the installation, I need to edit the /etc/hosts file of all the systems so that they can resolve each other. Edit the /etc/hosts
file on all the systems, as shown below.
geekflare@chef-geekflare:~$
127.0.0.1 localhost
127.0.1.1 geekflare
192.168.0.107 chef-geekflare
192.168.0.108 chef-workstation
192.168.0.109 client-node
I will run below command on all the three systems to update them.
geekflare@chef-geekflare:~$ sudo apt update
[sudo] password for geekflare:
Hit:1 http://security.ubuntu.com/ubuntu cosmic-security InRelease
Get:2 https://download.docker.com/linux/ubuntu bionic InRelease [64.4 kB]
Hit:3 http://ppa.launchpad.net/ansible/ansible/ubuntu cosmic InRelease
Hit:4 http://us.archive.ubuntu.com/ubuntu cosmic InRelease
Get:5 https://download.docker.com/linux/ubuntu bionic/stable amd64 Packages [9,594 B]
Hit:6 http://us.archive.ubuntu.com/ubuntu cosmic-updates InRelease
Hit:7 http://us.archive.ubuntu.com/ubuntu cosmic-backports InRelease
Get:8 http://apt.puppetlabs.com bionic InRelease [85.3 kB]
Get:9 http://apt.puppetlabs.com bionic/puppet6 amd64 Packages [32.4 kB]
Fetched 192 kB in 2s (84.6 kB/s)
Reading package lists... Done
Building dependency tree
Reading state information... Done
233 packages can be upgraded. Run 'apt list --upgradable' to see them.
Chef Server Installation
Chef Server is a component in the architecture that connects Workstation and Nodes. When the configurations are edited/changed on a workstation, they are pushed to the chef-server, and all the nodes pull these configuration changes from the Chef Server.
Now, let us run the command below to download the chef-server package.
geekflare@chef-geekflare:~$ wget https://packages.chef.io/files/stable/chef-server/13.0.17/ubuntu/18.04/chef-server-core_13.0.17-1_amd64.deb
--2019-10-23 04:04:35-- https://packages.chef.io/files/stable/chef-server/13.0.17/ubuntu/18.04/chef-server-core_13.0.17-1_amd64.deb
Saving to: ‘chef-server-core_13.0.17-1_amd64.deb’
chef-server-core_13 100%[===================>] 240.58M 1.33MB/s in 6m 16s
2019-10-23 04:10:51 (656 KB/s) - ‘chef-server-core_13.0.17-1_amd64.deb’ saved [252269838/252269838]
Now you need to run the following command install the chef server.
geekflare@chef-geekflare:~$ sudo dpkg -i chef-server-core_*.deb
chef-server-ctl
is command-line utility in chef-server. I will use this utility to start the chef-server services.
geekflare@chef-geekflare:~$ sudo chef-server-ctl reconfigure
Running handlers:
Running handlers complete
Chef Infra Client finished, 481/1028 resources updated in 04 minutes 08 seconds
Chef Server Reconfigured!
You can check the status of the services which have started using the below command.
geekflare@chef-geekflare:~$ sudo chef-server-ctl status
run: bookshelf: (pid 2452) 822s; run: log: (pid 29553) 951s
run: nginx: (pid 2318) 826s; run: log: (pid 30216) 908s
run: oc_bifrost: (pid 2296) 827s; run: log: (pid 29240) 996s
run: oc_id: (pid 2304) 826s; run: log: (pid 29308) 979s
run: opscode-erchef: (pid 2511) 822s; run: log: (pid 29707) 946s
run: opscode-expander: (pid 2416) 822s; run: log: (pid 29412) 958s
run: opscode-solr4: (pid 2393) 824s; run: log: (pid 29358) 964s
run: postgresql: (pid 2264) 827s; run: log: (pid 28769) 1021s
run: rabbitmq: (pid 3183) 792s; run: log: (pid 30476) 902s
run: redis_lb: (pid 30011) 926s; run: log: (pid 30010) 926s
Create User and Organization
Chef server connects workstation and client nodes. To link them, I will create an admin and organizer with their private keys.
Firstly, create a .chef directory to store the keys.
geekflare@chef-geekflare:~$ mkdir .chef
Now, I will use chef-server-ctl
to create a user. In the command below, chefadmin
is the user, Chef is the first name, GeekFlare is the last name, chefadmin@geekflare.com is the email id, geekflare is the password, chefadmin.pen is the RSA key.
geekflare@chef-geekflare:~$ sudo chef-server-ctl user-create chefadmin Chef GeekFlare chefadmin@geekflare.com 'geekflare' --filename ~/.chef/chefadmin.pem
Let us run a command to check the list of users on the chef server.
geekflare@chef-geekflare:~$ sudo chef-server-ctl user-list
chefadmin
pivotal
Now, I will use chef-server-ctl to create an organization. In the command below, chef-org is the organization name, Geekflare Chef Infrastructure is the full org name, chefadmin is the user we just created., chef-org.pem is the RSA key.
geekflare@chef-geekflare:~$ sudo chef-server-ctl org-create chef-org "Geekflare Chef Infrastructure" --association_user chefadmin --filename ~/.chef/chef-org.pem
Let us run a command to check the list of organizations on the chef server.
geekflare@chef-geekflare:~$ sudo chef-server-ctl org-list
chef-org
I have installed the chef-server completely, let us go ahead and install workstation where all the configurations are created.
Workstation
Workstation is the place where users create cookbooks. Cookbooks are nothing but the configuration units which are created to run specific tasks.
Let us run the command below to download the chef workstation package.
geekflare@chef-workstation:~$ wget https://packages.chef.io/files/stable/chef-workstation/0.2.43/ubuntu/18.04/chef-workstation_0.2.43-1_amd64.deb
--2019-10-23 05:37:41-- https://packages.chef.io/files/stable/chef-workstation/0.2.43/ubuntu/18.04/chef-workstation_0.2.43-1_amd64.deb
Resolving packages.chef.io (packages.chef.io)... 151.101.194.110, 151.101.130.110, 151.101.66.110, ...
Connecting to packages.chef.io (packages.chef.io)|151.101.194.110|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 129713682 (124M) [application/x-debian-package]
Saving to: ‘chef-workstation_0.2.43-1_amd64.deb’
chef-workstation_0.2.43-1_ 100%[=======================================>] 123.70M 2.37MB/s in 4m 25s
2019-10-23 05:42:18 (477 KB/s) - ‘chef-workstation_0.2.43-1_amd64.deb’ saved [129713682/129713682]
Let us run dpkg command to install a workstation on the ubuntu system.
geekflare@chef-workstation:~$ sudo dpkg -i chef-workstation_*.deb
Selecting previously unselected package chef-workstation.
(Reading database ... 273360 files and directories currently installed.)
Preparing to unpack chef-workstation_0.2.43-1_amd64.deb ...
Unpacking chef-workstation (0.2.43-1) ...
Setting up chef-workstation (0.2.43-1) ...
To run the experimental Chef Workstation App, use your
platform's package manager to install these dependencies:
libgconf-2.so.4 => not found
You can then launch the App by running 'chef-workstation-app'.
The App will then be available in the system tray.
Thank you for installing Chef Workstation!
You can find some tips on getting started at https://chef.sh/
I will now run a command to create a chef repository that will have all the cookbooks and other files.
geekflare@chef-workstation:~$ chef generate repo chef-repo
Recipe: code_generator::repo
* directory[/home/geekflare/chef-repo] action create
- create new directory /home/geekflare/chef-repo
* template[/home/geekflare/chef-repo/LICENSE] action create_if_missing
- create new file /home/geekflare/chef-repo/LICENSE
- update content in file /home/geekflare/chef-repo/LICENSE from none to 3c525c
(diff output suppressed by config)
* cookbook_file[/home/geekflare/chef-repo/.chef-repo.txt] action create_if_missing
- create new file /home/geekflare/chef-repo/.chef-repo.txt
- update content in file /home/geekflare/chef-repo/.chef-repo.txt from none to 2bed28
(diff output suppressed by config)
* cookbook_file[/home/geekflare/chef-repo/README.md] action create_if_missing
- create new file /home/geekflare/chef-repo/README.md
- update content in file /home/geekflare/chef-repo/README.md from none to 2b4f46
(diff output suppressed by config)
* cookbook_file[/home/geekflare/chef-repo/chefignore] action create_if_missing
- create new file /home/geekflare/chef-repo/chefignore
- update content in file /home/geekflare/chef-repo/chefignore from none to 9e2ffd
(diff output suppressed by config)
* remote_directory[/home/geekflare/chef-repo/cookbooks] action create_if_missing
- create new directory /home/geekflare/chef-repo/cookbooks
Recipe: code_generator::repo
* cookbook_file[/home/geekflare/chef-repo/cookbooks/README.md] action create_if_missing
- create new file /home/geekflare/chef-repo/cookbooks/README.md
- update content in file /home/geekflare/chef-repo/cookbooks/README.md from none to 54b03d
(diff output suppressed by config)
* execute[initialize-git] action run
- execute git init .
* template[/home/geekflare/chef-repo/.gitignore] action create_if_missing
- create new file /home/geekflare/chef-repo/.gitignore
- update content in file /home/geekflare/chef-repo/.gitignore from none to 11e5ee
(diff output suppressed by config)
Now I will create /chef-repo/.chef
directory, which will store all the knife configurations and the RSA keys.
geekflare@chef-workstation:~$ mkdir ~/chef-repo/.chef
geekflare@chef-workstation:~$ cd chef-repo/
Now let us generate RSA key-pair. We are generating this key to authenticate the workstation and get access to the chef server.
geekflare@chef-workstation:~/chef-repo$ ssh-keygen -b 4096
Generating public/private RSA key pair.
Enter file in which to save the key (/home/geekflare/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/geekflare/.ssh/id_rsa.
Your public key has been saved in /home/geekflare/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:oK/ZyLn+AOMj97F5Z0e1K5o1bxChyKx3ms4HvK06DxI geekflare@chef-geekflare
The key's randomart image is:
+---[RSA 4096]----+
| |
| . |
| o.. . . |
| .+.. . . |
| E .o S o . |
| . +..+ . o . |
|. = +..B .o. . |
| o =.&= =oooo |
| .&OB=oo o. |
+----[SHA256]-----+
Now copy the key to the chef-server from the workstation.
geekflare@chef-workstation:~/chef-repo$ sudo ssh-copy-id geekflare@192.168.0.107
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/home/geekflare/.ssh/id_rsa.pub"
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
geekflare@192.168.0.107's password:
Number of key(s) added: 1
Now try logging into the machine, with: "ssh 'geekflare@192.168.0.107'"
and check to make sure that only the key(s) you wanted were added.
Now I will copy the .pem files (chefadmin.pem and chef-org.pem) from the chef server to the workstation.
geekflare@chef-workstation:~/chef-repo$ scp geekflare@192.168.0.107:~/.chef/*.pem ~/chef-repo/.chef/
chefadmin.pem 100% 1674 105.1KB/s 00:00
chef-org.pem 100% 1674 103.0KB/s 00:00
Check if .pem files got copied successfully on the workstation.
geekflare@chef-workstation:~/chef-repo$ ls ~/chef-repo/.chef
chefadmin.pem chef-org.pem
Create Version Control
While working on a workstation, a lot of changes and editing happens in cookbooks (configuration units), so a version control system is required to keep track of these changes. So, let us go ahead and create a version control system using Git on the workstation. I will create a git repository inside the chef-repo directory. I will add a username and email to configure git
geekflare@chef-workstation:~$ git config --global user.name geekflare
geekflare@chef-workstation:~$ git config --global user.email avi@geekflare.com
Let us add the .chef directory to the .gitignore file.
geekflare@chef-workstation:~$ echo ".chef" > ~/chef-repo/.gitignore
Now I will run add and commit git commands from a chef-repo directory.
geekflare@chef-workstation:~$ cd ~/chef-repo
geekflare@chef-workstation:~/chef-repo$ git add .
geekflare@chef-workstation:~/chef-repo$ git commit -m "initial commit"
[master (root-commit) 99c8c11] initial commit
16 files changed, 359 insertions(+)
create mode 100644 .chef-repo.txt
create mode 100644 .gitignore
create mode 100644 LICENSE
create mode 100644 README.md
create mode 100644 chefignore
create mode 100644 cookbooks/README.md
create mode 100644 cookbooks/example/README.md
create mode 100644 cookbooks/example/attributes/default.rb
create mode 100644 cookbooks/example/metadata.rb
create mode 100644 cookbooks/example/recipes/default.rb
create mode 100644 data_bags/README.md
create mode 100644 data_bags/example/example_item.json
create mode 100644 environments/README.md
create mode 100644 environments/example.json
create mode 100644 roles/README.md
create mode 100644 roles/example.json
Check the status.
geekflare@chef-workstation:~/chef-repo$ git status
On branch master
nothing to commit, working tree clean
Generate Your First CookBook
Now the installation of WorkStation is complete, and you can start creating cookbooks on the WorkStation. Try generating a sample cookbook on the WorkStation and see if it gets generated successfully.
I will run the below command to generate a cookbook.
geekflare@chef-workstation:~$ Chef generate cookbook geekflare_cookbook
Generating cookbook geekflare_cookbook
- Ensuring correct cookbook file content
- Committing cookbook files to Git
- Ensuring delivery configuration
- Ensuring correct delivery build cookbook content
- Adding delivery configuration to feature branch
- Adding build cookbook to feature branch
- Merging delivery content feature branch to master
Your cookbook is ready. Type `cd geekflare_cookbook` to enter it.
There are several commands you can run to get started locally developing and testing your cookbook.
Type `delivery local --help` to see a full list.
Why not start by writing a test? Tests for the default recipe are stored at:
test/integration/default/default_test.rb
If you'd prefer to dive right in, the default recipe can be found at:
recipes/default.rb
Generate chef-repo, then move to a chef-repo directory
geekflare@chef-workstation:~$ Chef generate app chef-repo
WARNING: The command 'chef generator app' is deprecated and will be removed from the next major release of Chef DK / Workstation (April 2019)
Recipe: code_generator::app
* directory[/home/geekflare/chef-repo] action create (up to date)
* template[/home/geekflare/chef-repo/.kitchen.yml] action create
- create new file /home/geekflare/chef-repo/.kitchen.yml
- update content in file /home/geekflare/chef-repo/.kitchen.yml from none to ceae09
(diff output suppressed by config)
* directory[/home/geekflare/chef-repo/test/integration/default] action create
- create new directory /home/geekflare/chef-repo/test/integration/default
* template[/home/geekflare/chef-repo/test/integration/default/default_test.rb] action create_if_missing
- create new file /home/geekflare/chef-repo/test/integration/default/default_test.rb
- update content in file /home/geekflare/chef-repo/test/integration/default/default_test.rb from none to 0f757b
(diff output suppressed by config)
* template[/home/geekflare/chef-repo/README.md] action create
- update content in file /home/geekflare/chef-repo/README.md from 2b4f46 to 6401b8
(diff output suppressed by config)
* directory[/home/geekflare/chef-repo/cookbooks] action create (up to date)
* directory[/home/geekflare/chef-repo/cookbooks/chef-repo] action create
- create new directory /home/geekflare/chef-repo/cookbooks/chef-repo
* template[/home/geekflare/chef-repo/cookbooks/chef-repo/metadata.rb] action create
- create new file /home/geekflare/chef-repo/cookbooks/chef-repo/metadata.rb
- update content in file /home/geekflare/chef-repo/cookbooks/chef-repo/metadata.rb from none to e30be3
(diff output suppressed by config)
* cookbook_file[/home/geekflare/chef-repo/cookbooks/chef-repo/chefignore] action create
- create new file /home/geekflare/chef-repo/cookbooks/chef-repo/chefignore
- update content in file /home/geekflare/chef-repo/cookbooks/chef-repo/chefignore from none to 9e2ffd
(diff output suppressed by config)
* cookbook_file[/home/geekflare/chef-repo/cookbooks/chef-repo/Berksfile] action create
- create new file /home/geekflare/chef-repo/cookbooks/chef-repo/Berksfile
- update content in file /home/geekflare/chef-repo/cookbooks/chef-repo/Berksfile from none to 15e000
(diff output suppressed by config)
* directory[/home/geekflare/chef-repo/cookbooks/chef-repo/recipes] action create
- create new directory /home/geekflare/chef-repo/cookbooks/chef-repo/recipes
* template[/home/geekflare/chef-repo/cookbooks/chef-repo/recipes/default.rb] action create
- create new file /home/geekflare/chef-repo/cookbooks/chef-repo/recipes/default.rb
- update content in file /home/geekflare/chef-repo/cookbooks/chef-repo/recipes/default.rb from none to f56ecb
(diff output suppressed by config)
* directory[/home/geekflare/chef-repo/cookbooks/chef-repo/spec/unit/recipes] action create
- create new directory /home/geekflare/chef-repo/cookbooks/chef-repo/spec/unit/recipes
* cookbook_file[/home/geekflare/chef-repo/cookbooks/chef-repo/spec/spec_helper.rb] action create_if_missing
- create new file /home/geekflare/chef-repo/cookbooks/chef-repo/spec/spec_helper.rb
- update content in file /home/geekflare/chef-repo/cookbooks/chef-repo/spec/spec_helper.rb from none to 1f80e1
(diff output suppressed by config)
* template[/home/geekflare/chef-repo/cookbooks/chef-repo/spec/unit/recipes/default_spec.rb] action create_if_missing
- create new file /home/geekflare/chef-repo/cookbooks/chef-repo/spec/unit/recipes/default_spec.rb
- update content in file /home/geekflare/chef-repo/cookbooks/chef-repo/spec/unit/recipes/default_spec.rb from none to 666a01
(diff output suppressed by config)
* execute[initialize-git] action run
- execute git init .
* cookbook_file[/home/geekflare/chef-repo/.gitignore] action create
- update content in file /home/geekflare/chef-repo/.gitignore from 25558e to edcd62
(diff output suppressed by config)
Configure Knife
Knife is a command-line tool to manage nodes, cookbooks, and recipes. To configure Knife, create a config.rb file and put the below content in the file, these are knife configurations.
geekflare@chef-workstation:~$ sudo gedit ~/chef-repo/.chef/config.rb
current_dir = File.dirname(__FILE__)
log_level :info
log_location STDOUT
node_name 'chefadmin'
client_key "chefadmin.pem"
validation_client_name 'chef-org-validator'
validation_key "chef-org-validator.pem"
chef_server_url 'https://chef-geekflare/organizations/chef-org'
cache_type 'BasicFile'
cache_options( :path => "#{ENV['HOME']}/.chef/checksums" )
cookbook_path ["#{current_dir}/../cookbooks"]
Now go to a chef-repo directory and copy SSL certificates.
geekflare@chef-workstation:~$ cd chef-repo
geekflare@chef-workstation:~/chef-repo$ knife ssl fetch
WARNING: Certificates from chef-geekflare will be fetched and placed in your trusted_cert
directory (/home/geekflare/chef-repo/.chef/trusted_certs).
Knife has no means to verify these are the correct certificates. You should
verify the authenticity of these certificates after downloading.
Adding certificate for chef-geekflare in /home/geekflare/chef-repo/.chef/trusted_certs/chef-geekflare.crt
To check that config.rb is set correctly, run the command below.
geekflare@chef-workstation:~/chef-repo$ knife client list
chef-org-validator
Bootstrap a Node
Bootstrap runs from the workstation machine and installs the chef-client on the nodes. The nodes can then read configurations from the chef server by using the client node’s user and password to bootstrap a node.
I will now bootstrap a node with IP address 192.168.0.109, user name geekflare, and password geekflare.org.
geekflare@chef-workstation:~/chef-repo/.chef$ knife bootstrap 192.168.0.109 -x geekflare -P geekflare.org --node-name geekflare-client-1
Creating new client for geekflare-client-1
Creating new node for geekflare-client-1
Connecting to 192.168.0.109
192.168.0.109 -----> Installing Chef Omnibus (-v 14)
192.168.0.109 downloading https://omnitruck-direct.chef.io/chef/install.sh
192.168.0.109 to file /tmp/install.sh.9250/install.sh
192.168.0.109 trying wget...
192.168.0.109 ubuntu 18.10 x86_64
192.168.0.109 Getting information for chef stable 14 for ubuntu...
192.168.0.109 downloading https://omnitruck-direct.chef.io/stable/chef/metadata?v=14&p=ubuntu&pv=18.10&m=x86_64
192.168.0.109 to file /tmp/install.sh.9261/metadata.txt
192.168.0.109 trying wget...
192.168.0.109 sha1 534bae390bde3bd9d93bef99335f62246624f32b
192.168.0.109 sha256 94bc60b3a97ddadf77a70c7678ec77a676942c74f8152a2c70a0f5b68e22a42e
192.168.0.109 url https://packages.chef.io/files/stable/chef/14.14.25/ubuntu/18.04/chef_14.14.25-1_amd64.deb
192.168.0.109 version 14.14.25
192.168.0.109 downloaded metadata file looks valid...
192.168.0.109 downloading https://packages.chef.io/files/stable/chef/14.14.25/ubuntu/18.04/chef_14.14.25-1_amd64.deb
192.168.0.109 to file /tmp/install.sh.9261/chef_14.14.25-1_amd64.deb
192.168.0.109 trying wget...
192.168.0.109 Comparing checksum with sha256sum...
192.168.0.109 Installing chef 14
192.168.0.109 installing with dpkg...
192.168.0.109 Selecting previously unselected package chef.
(Reading database ... 204803 files and directories currently installed.)
192.168.0.109 Preparing to unpack .../chef_14.14.25-1_amd64.deb ...
192.168.0.109 Unpacking chef (14.14.25-1) ...
192.168.0.109 Setting up chef (14.14.25-1) ...
192.168.0.109 Thank you for installing Chef Infra Client! For help getting started visit https://learn.chef.io
192.168.0.109 Starting the first Chef Client run...
192.168.0.109 Starting Chef Client, version 14.14.25
192.168.0.109 resolving cookbooks for run list: []
192.168.0.109 Synchronizing Cookbooks:
192.168.0.109 Installing Cookbook Gems:
192.168.0.109 Compiling Cookbooks...
192.168.0.109 [2019-10-23T10:52:57-04:00] WARN: Node geekflare-client-1 has an empty run list.
192.168.0.109 Converging 0 resources
192.168.0.109
192.168.0.109 Running handlers:
192.168.0.109 Running handlers complete
192.168.0.109 Chef Client finished, 0/0 resources updated in 07 seconds
I will now list all the nodes which got bootstrapped
geekflare@chef-workstation:~/chef-repo/.chef$ knife node list
geekflare-client-1
Run below command to get the details of the node.
geekflare@chef-workstation:~/chef-repo/.chef$ knife node show geekflare-client-1
Node Name: geekflare-client-1
Environment: _default
FQDN: client-node
IP: 192.168.0.109
Run List:
Roles:
Recipes:
Platform: ubuntu 18.10
Tags:
Now the setup is ready!
We have successfully installed a chef server, workstation, and a node on Ubuntu. You can go ahead and start creating recipes and cookbooks in Chef for configuration management of infrastructure.
If you are an absolute beginner, then you may like taking this Udemy course too.
-
Avi is a tech enthusiast with expertise in trending technologies such as DevOps, Cloud Computing, Big Data and many more. He is passionate about learning cutting-edge technologies and sharing his knowledge with others through… read more