First, if you watched the debates last night and wonder if the world is ending, I want you to know, I sympathize with your feelings.
Never in my life have I seen my country facing down such a terrible dilemma. I don’t see a way out, one of them is going to win.
Many people will tell you otherwise, but left with a choice between evil, and more established evil, the answer is not to choose the lessor of two evils. You’re still selecting evil. Simply don’t vote.
Neither candidate is acceptable.
The American people didn’t prop up Trump. They didn’t prop up Hillary Clinton, either. Trump did. Clinton did. They did it in collaboration with each other for the sake of controlled opposition.
They did it by manipulation, coercion, bribery, and some allege even murder. The process being used to give you your options is no longer being used to give you an option, it’s being used as a platform to elevate the status of evil narcissists at the expense of every one of us, every one of you, every one of our children, every civilian man, woman and child in the war-torn countries already benefited by one of them, or possibly, the civilian men, women and children in the soon to be war-torn countries that are currently today untouched by the foreign policy of the other. Both want to expand our aggressions during a time when we should be receding our aggressions. The only winning move is not to play.
Simply do not vote. Yes, it’s your obligation as a citizen to vote. That’s why I’m saying it. People will remind you of that, too. And this is exactly why you should not vote if you recognize the choices being given to you as evil. Oh, and they are so terribly evil. One’s just been in the game a little longer than the other. Reject the vote. Reject the candidates. The next US President must never be taken seriously except as a threat to world peace, life, and liberty..
...aeternum.
So, political rant and declaration of conscientious objection to voting aside, with the last week or so of me being very, very tired due to odd hours I’ve been working, I’ve found myself freshly rested and really feeling quite lazy. I almost didn’t write this, but, I’ve been waiting for weeks now to finish putting this SDLC in place and I’m tired of coming across idea after idea for things I want to build and realizing I need to wait if I want to break the cowboy habit I’ve been performing with my dev projects.
That said, the number of my posts about setting up puppet master, what should have been easier to set up than anything else I’ve been or will be using this stack for, is turning out to rank up there with the number of unnecessary Batman sequels released in my lifetime.
And, it’s been just as frustrating for me as it has been for the various Batmen of my time:
Clearly Mr. Bale was trying to set up a Puppet Master with PuppetDB using the official docs when the samples for this were being created.
With the idea that maybe I was too tired to read right, I’ll now go back to where I left it the other night, rested, and finish the operation of building my preprod environment, which starts with puppet master.
At this point I have puppet, puppetserver, puppetDB, and puppetDB-termini installed, just none of it is configured.
Anywho, back to the docs: I’m on step 4:
https://docs.puppet.com/puppetdb/4.2/install_from_packages.html
Hilariously noted is that this is the official ‘Install from Packages’ doc for PuppetDB, with step 3 being to install PuppetDB via module and not package. I can’t elaborate on that without another rant. I am just confused. Surely this isn’t me.
Configuring the database.
Since it’s a fresh server still except for what’s documented in these articles, we’ll need the package “postgresql” install:
surro@oldhorse:~$ sudo apt-get install postgresql
[sudo] password for surro:
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
libpq5 libsensors4 postgresql-9.5 postgresql-client-9.5 postgresql-client-common postgresql-common postgresql-contrib-9.5 ssl-cert sysstat
Suggested packages:
lm-sensors postgresql-doc locales-all postgresql-doc-9.5 libdbd-pg-perl openssl-blacklist isag
The following NEW packages will be installed:
libpq5 libsensors4 postgresql postgresql-9.5 postgresql-client-9.5 postgresql-client-common postgresql-common postgresql-contrib-9.5 ssl-cert
sysstat
0 upgraded, 10 newly installed, 0 to remove and 43 not upgraded.
Need to get 4,839 kB of archives.
After this operation, 19.6 MB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 https://us.archive.ubuntu.com/ubuntu xenial-updates/main amd64 libpq5 amd64 9.5.4-0ubuntu0.16.04 [77.8 kB]
Get:2 https://us.archive.ubuntu.com/ubuntu xenial/main amd64 libsensors4 amd64 1:3.4.0-2 [28.4 kB]
Get:3 https://us.archive.ubuntu.com/ubuntu xenial/main amd64 postgresql-client-common all 173 [28.3 kB]
Get:4 https://us.archive.ubuntu.com/ubuntu xenial-updates/main amd64 postgresql-client-9.5 amd64 9.5.4-0ubuntu0.16.04 [860 kB]
Get:5 https://us.archive.ubuntu.com/ubuntu xenial/main amd64 ssl-cert all 1.0.37 [16.9 kB]
Get:6 https://us.archive.ubuntu.com/ubuntu xenial/main amd64 postgresql-common all 173 [153 kB]
Get:7 https://us.archive.ubuntu.com/ubuntu xenial-updates/main amd64 postgresql-9.5 amd64 9.5.4-0ubuntu0.16.04 [2,980 kB]
Get:8 https://us.archive.ubuntu.com/ubuntu xenial/main amd64 postgresql all 9.5+173 [5,336 B]
Get:9 https://us.archive.ubuntu.com/ubuntu xenial-updates/main amd64 postgresql-contrib-9.5 amd64 9.5.4-0ubuntu0.16.04 [446 kB]
Get:10 https://us.archive.ubuntu.com/ubuntu xenial-updates/main amd64 sysstat amd64 11.2.0-1ubuntu0.1 [243 kB]
Fetched 4,839 kB in 3s (1,340 kB/s)
Preconfiguring packages ...
Selecting previously unselected package libpq5:amd64.
(Reading database ... 99646 files and directories currently installed.)
Preparing to unpack .../libpq5_9.5.4-0ubuntu0.16.04_amd64.deb ...
Unpacking libpq5:amd64 (9.5.4-0ubuntu0.16.04) ...
Selecting previously unselected package libsensors4:amd64.
Preparing to unpack .../libsensors4_1%3a3.4.0-2_amd64.deb ...
Unpacking libsensors4:amd64 (1:3.4.0-2) ...
Selecting previously unselected package postgresql-client-common.
Preparing to unpack .../postgresql-client-common_173_all.deb ...
Unpacking postgresql-client-common (173) ...
Selecting previously unselected package postgresql-client-9.5.
Preparing to unpack .../postgresql-client-9.5_9.5.4-0ubuntu0.16.04_amd64.deb ...
Unpacking postgresql-client-9.5 (9.5.4-0ubuntu0.16.04) ...
Selecting previously unselected package ssl-cert.
Preparing to unpack .../ssl-cert_1.0.37_all.deb ...
Unpacking ssl-cert (1.0.37) ...
Selecting previously unselected package postgresql-common.
Preparing to unpack .../postgresql-common_173_all.deb ...
Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common'
Unpacking postgresql-common (173) ...
Selecting previously unselected package postgresql-9.5.
Preparing to unpack .../postgresql-9.5_9.5.4-0ubuntu0.16.04_amd64.deb ...
Unpacking postgresql-9.5 (9.5.4-0ubuntu0.16.04) ...
Selecting previously unselected package postgresql.
Preparing to unpack .../postgresql_9.5+173_all.deb ...
Unpacking postgresql (9.5+173) ...
Selecting previously unselected package postgresql-contrib-9.5.
Preparing to unpack .../postgresql-contrib-9.5_9.5.4-0ubuntu0.16.04_amd64.deb ...
Unpacking postgresql-contrib-9.5 (9.5.4-0ubuntu0.16.04) ...
Selecting previously unselected package sysstat.
Preparing to unpack .../sysstat_11.2.0-1ubuntu0.1_amd64.deb ...
Unpacking sysstat (11.2.0-1ubuntu0.1) ...
Processing triggers for libc-bin (2.23-0ubuntu3) ...
Processing triggers for man-db (2.7.5-1) ...
Processing triggers for systemd (229-4ubuntu7) ...
Processing triggers for ureadahead (0.100.0-19) ...
Setting up libpq5:amd64 (9.5.4-0ubuntu0.16.04) ...
Setting up libsensors4:amd64 (1:3.4.0-2) ...
Setting up postgresql-client-common (173) ...
Setting up postgresql-client-9.5 (9.5.4-0ubuntu0.16.04) ...
update-alternatives: using /usr/share/postgresql/9.5/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode
Setting up ssl-cert (1.0.37) ...
Setting up postgresql-common (173) ...
Adding user postgres to group ssl-cert
Creating config file /etc/postgresql-common/createcluster.conf with new version
Creating config file /etc/logrotate.d/postgresql-common with new version
Building PostgreSQL dictionaries from installed myspell/hunspell packages...
Removing obsolete dictionary files:
Setting up postgresql-9.5 (9.5.4-0ubuntu0.16.04) ...
Unescaped left brace in regex is deprecated, passed through in regex; marked by <-- HERE in m/(?<!\\)\${ <-- HERE ([^}]+)}/ at /usr/sbin/pam_getenv line 78.
Unescaped left brace in regex is deprecated, passed through in regex; marked by <-- HERE in m/(?<!\\)\${ <-- HERE ([^}]+)}/ at /usr/sbin/pam_getenv line 78.
Unescaped left brace in regex is deprecated, passed through in regex; marked by <-- HERE in m/(?<!\\)\${ <-- HERE ([^}]+)}/ at /usr/sbin/pam_getenv line 78.
Unescaped left brace in regex is deprecated, passed through in regex; marked by <-- HERE in m/(?<!\\)\${ <-- HERE ([^}]+)}/ at /usr/sbin/pam_getenv line 78.
Unescaped left brace in regex is deprecated, passed through in regex; marked by <-- HERE in m/(?<!\\)\${ <-- HERE ([^}]+)}/ at /usr/sbin/pam_getenv line 78.
Unescaped left brace in regex is deprecated, passed through in regex; marked by <-- HERE in m/(?<!\\)\${ <-- HERE ([^}]+)}/ at /usr/sbin/pam_getenv line 78.
Unescaped left brace in regex is deprecated, passed through in regex; marked by <-- HERE in m/(?<!\\)\${ <-- HERE ([^}]+)}/ at /usr/sbin/pam_getenv line 78.
Unescaped left brace in regex is deprecated, passed through in regex; marked by <-- HERE in m/(?<!\\)\${ <-- HERE ([^}]+)}/ at /usr/sbin/pam_getenv line 78.
Unescaped left brace in regex is deprecated, passed through in regex; marked by <-- HERE in m/(?<!\\)\${ <-- HERE ([^}]+)}/ at /usr/sbin/pam_getenv line 78.
Unescaped left brace in regex is deprecated, passed through in regex; marked by <-- HERE in m/(?<!\\)\${ <-- HERE ([^}]+)}/ at /usr/sbin/pam_getenv line 78.
Unescaped left brace in regex is deprecated, passed through in regex; marked by <-- HERE in m/(?<!\\)\${ <-- HERE ([^}]+)}/ at /usr/sbin/pam_getenv line 78.
Unescaped left brace in regex is deprecated, passed through in regex; marked by <-- HERE in m/(?<!\\)\${ <-- HERE ([^}]+)}/ at /usr/sbin/pam_getenv line 78.
Unescaped left brace in regex is deprecated, passed through in regex; marked by <-- HERE in m/(?<!\\)\${ <-- HERE ([^}]+)}/ at /usr/sbin/pam_getenv line 78.
Unescaped left brace in regex is deprecated, passed through in regex; marked by <-- HERE in m/(?<!\\)\${ <-- HERE ([^}]+)}/ at /usr/sbin/pam_getenv line 78.
Unescaped left brace in regex is deprecated, passed through in regex; marked by <-- HERE in m/(?<!\\)\${ <-- HERE ([^}]+)}/ at /usr/sbin/pam_getenv line 78.
Creating new cluster 9.5/main ...
config /etc/postgresql/9.5/main
data /var/lib/postgresql/9.5/main
locale en_US.UTF-8
socket /var/run/postgresql
port 5432
update-alternatives: using /usr/share/postgresql/9.5/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode
Setting up postgresql (9.5+173) ...
Setting up postgresql-contrib-9.5 (9.5.4-0ubuntu0.16.04) ...
Setting up sysstat (11.2.0-1ubuntu0.1) ...
Creating config file /etc/default/sysstat with new version
update-alternatives: using /usr/bin/sar.sysstat to provide /usr/bin/sar (sar) in auto mode
Processing triggers for libc-bin (2.23-0ubuntu3) ...
Processing triggers for systemd (229-4ubuntu7) ...
Processing triggers for ureadahead (0.100.0-19) ...
surro@oldhorse:~$
So far so good. This ought to be fun as I’ve not messed with postgresql much. I’ve played around with redis, oracle db, mysql, MS Access, MS SQL Server, couchbase, memcache, documentum, et al, but almost not at all with postgresql, so, troubleshooting should be fun.
That’s entirely my fault though, postgresql is a widely used database that I should already have mastery of at this point in the game, so I will need to learn it to avoid crises eventually.
And, of course, the instructions are wrong. They inappropriately tell you to go to a root shell, to run the following:
$ sudo -u postgres sh $ createuser -DRSP puppetdb $ createdb -E UTF8 -O puppetdb puppetdb $ exit
The -u postgres sh arguments to sudo make you call sh as the new postgres user, which will not have access to the directories you’re modifying if you become root first. You should, however, use sudo as a user with sudoers access.
Flawless, as not root:
surro@oldhorse:~$ sudo -u postgres sh $ createuser -DRSP puppetdb Enter password for new role: Enter it again: $ createdb -E UTF8 -O puppetdb puppetdb $ exit surro@oldhorse:~$
Maybe they meant in the context of running in a postgresql service and not the system-level server. Since this is all in the context of installing system-level services, using the context of a subsystem or service is very confusing.
If you’re not running an existing postgresql service then run the following to turn on the regexp-optimized indexing extension pg_trgm, which they probably meant to say is an optional performance enhancement:
$ sudo -u postgres sh $ psql puppetdb -c 'create extension pg_trgm' $ exit
Here goes:
surro@oldhorse:~$ sudo -u postgres sh [sudo] password for surro: $ psql puppetdb -c 'create extension pg_trgm' CREATE EXTENSION $ exit surro@oldhorse:~$
At this point, a me who hasn’t been fucking with this for 3 weeks already should stop and go see what that did. I genuinely have no idea what the fuck that just did.
I have no idea what that just did. I should probably figure out how extensions work in postresql after all this.
Next, we will ‘most likely’ need to modify the pg_hba.conf file to allow for MD5 authentication from at least localhost.
“Most likely” surely means people are waiting to be alerted that you didn’t do this. To make fun of you.
Since the guide didn’t start with any architecture breakdown about what’s going on, it looks like we’re just doing the steps blindly and deciphering what’s going on with the clues they give you as you read. Instead of providing a setup script like most database backend tools do [link to one I created just for this using the new pipeline when this is done] so that you don’t have to be able to build the thing you’re installing, which meets the purpose of distribution of software built by professionals instead of copies of K&R containing hand-written inspirational quotes on the inside covers.
They inappropriately tell you to find this file by using locate, but locate uses a database (oh, the irony of the context here), which is updated by a script called updatedb, which absolutely has to run before it’ll see the file. Some distributions have a cron job which regularly updates this, if yours doesn’t, you should set it to run at a good time for some disk load. But my point here is that locate will absolutely not have your pg_hba.conf in it’s database yet. The alternative provided here is to consult my distribution’s documentation for the PostgreSQL confdir. Since I don’t know or care what that is right now I’m running updatedb with sudo because the database my locate uses is system-wide and it uses permission checks to determine if a result is stored or displayed.
surro@oldhorse:~$ sudo updatedb
[sudo] password for surro:
surro@oldhorse:~$ locate pg_hba.conf
/etc/postgresql/9.5/main/pg_hba.conf
/usr/share/postgresql/9.5/pg_hba.conf.sample
surro@oldhorse:~$
Woot, there’s the live copy and a sample. They are somewhat similar:
surro@oldhorse:~$ sudo diff /etc/postgresql/9.5/main/pg_hba.conf /usr/share/postgresql/9.5/pg_hba.conf.sample
75,85c75
<
<
<
< # DO NOT DISABLE!
< # If you change this first entry you will need to make sure that the
< # database superuser can access the database using some other method.
< # Noninteractive access to all databases is required during automatic
< # maintenance (custom daily cronjobs, replication, and similar tasks).
< #
< # Database administrative login by Unix domain socket
< local all postgres peer
---
> @authcomment@
89,90c79,80
< # "local" is for Unix domain socket connections only
< local all all peer
---
> @remove-line-for-nolocal@# "local" is for Unix domain socket connections only
> @remove-line-for-nolocal@local all all @authmethodlocal@
92c82
< host all all 127.0.0.1/32 md5
---
> host all all 127.0.0.1/32 @authmethodhost@
94c84
< host all all ::1/128 md5
---
> host all all ::1/128 @authmethodhost@
97,99c87,89
< #local replication postgres peer
< #host replication postgres 127.0.0.1/32 md5
< #host replication postgres ::1/128 md5
---
> @remove-line-for-nolocal@#local replication @default_username@ @authmethodlocal@
> #host replication @default_username@ 127.0.0.1/32 @authmethodhost@
> #host replication @default_username@ ::1/128 @authmethodhost@
So, something of a float ring if this gets derailed into the weird PostgreSQL abyss in my blind spot.
We want to enable MD5 Authentication, ultimately. If it follows most patterns it’ll store an MD5 hash of the password, or a weird combination of user handle and password, and compare a hash of what the user provides during session authentication to what is stored in the database. It is a simple, secure, and very effective authentication method.
Drunk Boss Mode:
sudo vim $(locate pg_hba.conf | grep -E "pg_hba\.conf$" | head -1)
You’ll see a line like this (for me it was line 90):
local all all peer
And you’ll change it to this:
local all all md5
Which is all the configuration you needed for all that explanation. Change “peer” to “md5” on line 90.
Restart postgresql:
surro@oldhorse:~$ sudo service postgresql restart
Time to test:
surro@oldhorse:~$ psql -h localhost puppetdb puppetdb Password for user puppetdb: psql (9.5.4) SSL connection (protocol: TLSv1.2, cipher: ECDHE-RSA-AES256-GCM-SHA384, bits: 256, compression: off) Type "help" for help. puppetdb=>
Stuff’s good.
Until you look at the PuppetDB doc. This is some poorly thought out slosh, man:
To configure PuppetDB to use this database, put the following in the
[database]section:subname = //<HOST>:<PORT>/<DATABASE> username = <USERNAME> password = <PASSWORD>
You need to identify where. Put the following in the [database] section where? What file are they referring to? It just jumps right in without saying or linking to where someone might find out.
Ok. So remember all the user details and database identifiers you just set up so you can put them in what I assume to be the init script config file for PuppetDB which is earlier in the documentation but needed to be skipped to get to this point in a linear fashion.
And now for something completely different.
If you’ve installed PuppetDB from a package, by default it will use the conf.d config style. The default config directory is /etc/puppetlabs/puppetdb/conf.d. If you’re running from source, you may use the -c command-line argument to specify your config file or directory.
We installed by the module, but haven’t changed anything, so according to the docs (lol), it should be in a default location at /etc/puppetlabs/puppetdb/conf.d/:
root@oldhorse:~# ls /etc/puppetlabs/ code mcollective puppet puppetserver pxp-agent root@oldhorse:~# ls /etc/puppetlabs/puppetserver/ conf.d logback.xml request-logging.xml services.d root@oldhorse:~#
Nope. What a nightmare. Can someone please email me to let me know what puppet labs’ definition of the word “default” is?
I have assumed incorrectly and need to backtrack a bit. The init script config file only specifies the location of the puppetdb config file:
root@oldhorse:~# locate puppetdb | grep etc | grep -v code
/etc/default/puppetdb
/etc/logrotate.d/puppetdb
root@oldhorse:~#
Ah. So here is where we’d put their default path, and create an /etc/puppetlabs/puppetdb/conf.d directory. /etc/puppetlabs/puppetdb does not exist because we installed via module, remember? That path only existed when you installed from package. Starting to think the advice I got on IRC the other night was bogus, as I was told the module installation path did alot of autoconfiguration while the package installation was hands on, while here we are having to create the entire /etc directory tree and even set up basic authentication, create databases, tie config files together with that non-existent tree, and anything else I don’t know about yet.
DBM:
root@oldhorse:~# vim $(locate puppetdb | grep etc | grep -v code | head -1)
Indeed, on line 15, it is set to /etc/puppetlabs/puppetdb/conf.d:
So they’re not just randomly making stuff up at least.
I’ve used up my patience for this for the night. I’m going to bed. It looks like we’re getting a 4th Episode.
https://docs.puppet.com/puppetdb/4.2/configure.html#the-puppetdb-configuration-files