docwhat's avatardocwhat's blog

User credentials in a Jenkins pipeline

The Credentials User Guide implies you can use user credentials. These are credentials that are owned and managed by a user instead of the administrator or the owner of a folder.

The guide explains how to use them with Freestyle jobs (classic Jenkins jobs) but not how to use them with the newer declarative pipeline jobs.

In this post, I’ll show you how to:

  • Set up a job to ask for credentials when triggered.
  • Allow users to provide their user credentials as a parameter.
  • Configure the pipeline job to use the passed in credentials.
Read More…

DNS not working with Go binaries and VPNs

If you are having DNS problems (i.e., Host not found) while using a Go binary while connected via a VPN… then I have a solution for you.

The solution is dns-heaven. Just use this command if you trust shell scripts running as root from random people on the Internet:

sudo -v \
  && curl -L \
   | sudo bash

What’s going on?

On macOS, if a Go program is compiled with CGO_ENABLED=0 then Go uses its own internal network name resolver. This resolver only knows about /etc/resolv.conf and doesn’t know about the libSystem (macOS libc) library and its name resolution functions.

macOS (like most modern OSes) has smarter DNS lookups than just using an /etc/resolv.conf which allow it to smoothly handle switching networks.

When a VPN is being used then DNS lookups will be split between the VPN’s DNS servers and your ISP’s DNS servers, depending on the hostname.

You can setup custom name servers for certain domains by creating resolv.conf style files in /etc/resolver/<domain>. This is what your VPN software is doing behind the covers.

This problem is VPN agnostic. In my case, I’m using “Cisco AnyConnect” but the same problem will exist with any VPN software.

It looks like this will be fixed in Go version 1.13, commit f20b42a landed in master branch early April, 2018.

See also:

Read More…

Rules for a good announcement

Writing for the digital medium (email, blog posts, slack, discord, etc.) requires breaking the rules you learned in school.

I’m going to explain some rules I use to create announcements for email and slack. These rules work for other digital mediums as well.

To be clear; By announcement I mean notices saying there will be an outage or an upgrade, etc.

Read More…

Vim: indirect variable access

In Vim you can use a variable as a variable name.

Instead of using the variable name directly, wrap it in curly braces ({}). This will use the contents of the variable as the variable name.

let l:variable_name = 'b:infosec_username'
let {l:variable_name} = 'George'

echo exists(l:variable_name) " => 1
echo {l:variable_name} " => George
Read More…

El Capitan and the evils of OpenSSL

Are you having trouble with SSL on El Capitan (OS X 10.11)?

Me too.

Read More…

Spread your knowledge

I found myself trying to figure out how to disable something called NeoComplete (previously known as NeoComplCache) when editing markdown in Vim.

It was colliding with my Markdown stuff pretty badly and had really bad suggestions anyway (I mean, I’m writing text, not code… so no surprise) and I was getting annoyed of turning it off by hand.

So I Googled for a solution and I found someones .vimrc and while it was written for NeoComplCache, it was easy to just change all the NeoComplCache strings to NeoComplete.

My .vimrc actually tries to use NeoComplete and if I can’t run it (i.e. Vim isn’t new enough or doesn’t have Lua compiled in) then I fall back to NeoComplCache.

So I went to check if I had similar code for NeoComplCache… and I did. In fact, it was the same code.

“Huh”, I thought, “I must’ve borrowed this from the same .vimrc earlier”

I double checked the URL of the .vimrc I was borrowing code from… and it was my .vimrc.

Some co-workers were trying to reimplement our builds to use CMake. They were having trouble using xsltproc to “compile” files (e.g. convert them from .xml to something else).

Guess what they found.

I used to work with the talented Carol. She left the company and went to work elsewhere.

When I next saw her, she stomped up to me and complained:

Whenever I Google for the answer to something, you’re there! Question on StackOverflow? You answered it. I go to file a bug on GitHub, you’ve already filed it or commented on it. Why won’t you go away?!?!

She was kidding… I think.

But the point is, I spend a lot of time contributing back to the community. Not necessarily by writing code (I do that, too) but by filing bugs, answering questions, adding helpful comments to issues, etc.

It isn’t that hard to write a bug, answer a question, etc. and it can be really helpful for others… or drive them mad.

Read More…

Chef, Puppet, Heat, Juju, Docker, etc.

Someone emailed me with this question:

I am interested in learning different orchestration mechanisms and would like to understand how they differ.

What are the differences between Chef, Puppet, Heat/HOT, Juju, and Docker?

When would I use a specific one?

There seems to be a lot of similarity between these.

Since I get these kind of questions a lot, I thought I would write up what I know of these (in some cases, little).


These are tools for configuring your server. Some run once and are never seen again, others run continuously and update the state as requirements change, most can be run in either mode.

All these tools can work with other things listed here. For example, I use Chef to deploy applications in Docker containers.


CFEngine was created in 1993 as an Open Source project. There is a company around it (named CFEngine) that does paid support, but you don’t have to pay anything unless you want to.

I haven’t used it but it has its own syntax that made it seem less attractive to me. It’s also a first-gen provisioning system. It’s written in C, and is fast. But I haven’t met anyone who uses CFEngine who configures as much as I’ve seen with an average Puppet or Chef user.


Puppet was created in 2005. It’s open source and built mostly on top of Ruby.

Puppet was obviously an answer to problems seen in CFEngine combined with a desire to use Ruby.

It requires a central server (this may have changed recently) to coordinate secrets, configurations and recipes.

It uses its own language for describing states and adding new functionality can be difficult. A puppet process runs on the system to maintain the state of the server, or make changes overtime (as you push changes to the central server.

It is also very flexible, and until recently, didn’t offer much assistance in how to organize your configuration files, etc.

In the past, when you wanted to work on a recipe, you had to try it out on a live system. This was problematic in my group, where developers wanted to be able to modify and assist with writing rules.

There was a like to about Puppet but ultimately, we switched to Chef.

The Puppet language is annoying. Writing a good language/syntax from scratch is hard and I never felt that the Puppet group was paying enough attention to it.

Puppet (at the time, I think this is changing) had no way to write tests or to test a recipe without trying it out on a live server. This made things really scary when we were making big change.

We also couldn’t give developers access to our recipes without giving away our secrets (passwords, etc.) That hindered our ability to move quickly.

Puppet uses a pull model. A process runs all the time and applies the rules every so often by asking the server what rules to run.


Chef was created in 2008. It is Open Source and is written in ruby (and parts of the server are in Erlang).

Chef was created when some of the developers for Puppet split away and wrote a whole new system, creating Chef.

Chef can be run with a central server (a chef-server) which can hold secrets, recipes, etc. and coordinate all the server.

Chef could also be run in a stand-alone mode using chef-zero, chef-solo, or chef-apply.

Chef has several well designed testing tools, including tools to bring up a VM on your workstation running the recipes under test.

Chef is spending a lot of effort to improve the experience for people writing new recipes, including a ChefDK that installs everything you need to get started writing recipes.

Chef also does a really good job separating secrets from non-secrets, which means our developers can actually talk to our chef-server without us worrying about them getting access to passwords, etc.

A lot of cloud companies that use Chef use chef-zero to pre-create VMs, then delete chef, and then make the system live. They then dispose of the VM when they need configuration changes.

Where I work, we manage lots of resources for developers and our systems are more permanent. So we use chef-server to monitor all the systems.

In addition, we use Chef to not only configure our systems to be in line with our security policies, but we use it to actually audit our systems as well!

Chef is normally a pull-model (it has a process that runs continuously that polls the chef-server) but with the Enterprise license you get a push ability too.


Ansible was created in 2012 is also Open Source.

I don’t know much about it, but it actually sounds pretty good. It is new and I believe it has taken away a lot of the lessons learned in Puppet and Chef.

It ships with Fedora, I don’t know what RedHat’s plans are for it.

JuJu (Ubuntu)

I haven’t used JuJu but my impression is that is like a “meta” package manager with some configuration built-in.

I don’t think it is as complete as the other systems above, but I think the other systems could make use of JuJu… as well as JuJu could be used to set up the above systems.

It only runs on Ubuntu as far as I know.


Isolating your application setup, deployment and configuration can make your life much easier. It can help to reliably reproduce working applications across your network as well as find bugs and do high quality QA.


You create a container image with your application all ready setup except for certain values which you configure via environment variables.

A Docker container should only contain the processes needed for the application, and nothing more. Docker containers can only be run on linux servers.

Docker containers can “stack” and communicate among them selves to allow building stacks of intercommunicating applications.

I like Docker a lot and it makes a really good way to pass around applications because you know that it’ll run the same in QA, Production, and for developers when debugging problems.

You can manage docker via any of the system provisioning systems above.


There are more infrastructure provisioning systems out there, but our setup hasn’t become so complicated (yet) that we have needed them.


Heat is a part of OpenStack and is used to provision infrastructure: networks, VMs, routes, etc. Heat can use some kind of system provisioning tools like Chef and Puppet to setup the VMs, or you can use pre-built VM images.

You could, for example, write a template describing a load-balancing setup with several apps and a private network connecting them. You could then either scale one setup using the template (e.g. increasing load-balancers, replicated dbs, etc.) or quickly create a new setup of server (e.g. a staging or development setup). Chef has something similar called chef-metal, though it isn’t version 1.0.0 yet (but it works).

Chef Provisioning (was Chef Metal)

Chef Provisioning is a system that mainly works with VMs. It can work with the surrounding infrastructure, depending on your setup.

It can work with OpenStack, AWS, Azure, and several other cloud providers.

As the name implies it integrates well with Chef, but could be used with any of the other systems above.

My understanding is that while it works, it isn’t mature yet.


Read More…

Setting your environment in test-kitchen

When using test-kitchen it may be necessary to set the environment of your nodes.

You can do with by changing your .kitchen.yml file. In my example, I’ll show it at the root, but they can be set on a per-suite level as well, which is handy to test different environments.

Read More…

Unindenting HEREDOCs in Ruby

This is a bit of code I wanted to save.

When using HEREDOCs in Ruby, the <<- operator is handy to keep everything indented in the source. But it doesn’t help with the content of the HEREDOC.


def example
  puts <<-EOF
This is left.

  This is indented two.

In rails, you can do this:

def example
  puts <<-EOF.strip_heredoc
    This is left.

      This is indented two.

There’s a helpful Stack Overflow question on this, in fact.

Here’s a simplish solution for plain ‘ol Ruby:

def unindent(string)
  first = string[/As*/]
  string.gsub(/^#{Regexp.quote first}/, '')

def example
  puts unindent(<<-EOF)
    This is flush left.

      This is indented by two spaces

Too bad you can’t pull in some of these Rails monkey patches without pulling in lots of stuff you don’t want.

Read More…

How to rename a Chef node

In Chef the node_name is for human usage. By default it is set to the fqdn. Which is annoying for typing.

In my network, all hosts have the same domain name. However, we knife bootstraped this one system without setting the node name with the -N flag.

Therefore I wanted to rename the nodes. With some experimentation, I figured it out.

Read More…
See all blog posts…π