Category: Computer

Authenticate through Client Certificate

This post documents how to setup using a client certificate in accessing a private web page.

The involved components are as follows:

  • Web server: Apache 2.4
  • Browser: Firefox 98

Steps:

  1. Generate CA certificate: see the previous post
  2. Configure web server to require client certificates signed by the CA
    • Add SSLVerifyClient require to the protected Location
    • Add SSLCACertificateFile to point to the CA certificate
  3. Generate a personal certificate: also described in the previous post
  4. Sign the personal certificate with CA certificate: ditto
  5. Import personal certificate into browser
    Settings → Privacy & Security → Certificates → View Certificates… → Your Certificates → Import…
  6. Configure browser to enable post-handshake authentication (Only needed if you get “…Cannot perform Post-Handshake Authentication” error. Usually happens when securing only a sub path instead of the entire server.)
    For Firefox (98)
    1. Go to the URL about:config
    2. Find and enable security.tls.enable_post_handshake_auth
  7. All set! Go ahead and test it out…

Self-signed CA certificate

It’s useful for signing certificates for internal use. Here is a quick way for setting up a self-signed CA using the openssl command in Linux.

mkdir ca
cd ca
openssl req -nodes -new -x509 -keyout ca.key -out ca.crt -subj "/C=TW/ST=Taiwan/L=Taipei/O=ccdw.org/OU=sam/CN=sam.ccdw.org/emailAddress=root@sam.ccdw.org"

Server certificate

To create a server certificate signing request along with a new server key:

openssl req -nodes -new -keyout server.key -out server.csr -subj "/C=TW/ST=Taiwan/L=Taipei/O=ccdw.org/OU=server/CN=server.ccdw.org/emailAddress=server@ccdw.org"

To sign the server certificate:

openssl x509 -req -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out server.crt -days 3650

The resulting key can certificate of the server can be combine into a PEM file:

cat server.key server.crt > server.pem

Client certificate

To create a client certificate signing request along with a new client key:

openssl req -nodes -new -keyout client.key -out client.csr -subj "/C=TW/ST=Taiwan/L=Taipei/O=ccdw.org/OU=client/CN=Good Client/emailAddress=good@client.ccdw.org"

To sign the client certificate:

openssl x509 -req -in client.csr -CA ca.crt -CAkey ca.key -CAserial ca.srl -out client.crt -days 1000

The browser usually needs the certificate in PKCS#12 format. To convert the format of the certificate:

openssl pkcs12 -export -out client.pfx -inkey client.key -in client.crt

The .pfx file can then be imported into a browser.

Bash command line menu system

For trained users, command-line interface of shells, such as BASH, is very convenient for crafting up complex actions that are much more cumbersome to do in GUI. However, some action may require a long line of command that requires looking up documentation to construct. One would find themself using “history” to look up a past invocation for copying to the current line of command. It is thus useful to have a command menu system for these reusable command lines available. choice.sh is my solution to this need. The content of the script is as follows:

#!/bin/bash
# Read and list lines from "choice.cmd". Allowing user interaction
# through stderr to choose a line and pass it back to stdout so it
# can be "eval"ed by the BASH

mapfile -u 4 -t 4<${BASH_SOURCE[0]%\.sh}.cmd

echo Run command: >&2
for ((i=0;i<${#MAPFILE[@]};i++));do
	printf "%4s: %s\n" "$((i+1))" "${MAPFILE[$i]}" >&2
done
printf "Choice: " >&2
read i
re='^[1-9][0-9]*$'
if [[ $i =~ $re ]] && [[ $((i-1)) -lt ${#MAPFILE[@]} ]]; then
	echo ${MAPFILE[$((i-1))]}
else
	echo "echo Cancelled."
fi

One can place file in, say, ~/lib, and add an alias, alias c='eval `~/lib/choice.sh`' to the ~/.bashrc. The menu system can then be invoked by typing c, enter on the command line. It will read all lines from the file choice.cmd in the same directory as the script, list them to the terminal screen using stderr, allow the user to make a choice, and pass the line to stdout so it will be evaled by the shell. Example usage is as follows:

cjj@parakeet:~$ c
Run command:
   1: eval "$(/home/cjj/opt/miniconda3/bin/conda shell.bash hook)"
   2: xrandr --output HDMI-1 --mode 1024x768 --scale 1x1 --pos 0x312
   3: xrandr --output HDMI-1 --mode 1024x768 --scale-from 1920x1080 --same-as eDP-1
Choice: 1
(base) cjj@parakeet:~$

Fetch images for html

Find img tags with remote source images in an HTML file and download the images to a local folder. Change the HTML files to use the local files instead.

This was done with the one liner:

grep img rbm.html|tr '"' '\n'|grep http|sort -u|while read f;do l=`basename ${f}`;ff=${l%.???};fe=${l##*\.};echo ${ff} ${fe};if [ -f "rbm_pics/${l}" ];then i=1;while [ -f "rbm_pics/${ff}-${i}.${fe}" ];do i=$((i+1));done;l="${ff}-${i}.${fe}";fi;curl --output rbm_pics/${l} "${f}";sed "sX${f}Xrbm_pics/${l}X" rbm0.html > t.html;mv t.html rbm0.html;done

A better formatted version is as follows:

grep img rbm.html|tr '"' '\n'|grep http|sort -u|while read f;do
	l=`basename ${f}`
	ff=${l%.???}
	fe=${l##*\.}
	echo ${ff} ${fe}
	if [ -f "rbm_pics/${l}" ];then
		i=1
		while [ -f "rbm_pics/${ff}-${i}.${fe}" ];do
			i=$((i+1))
		done
		l="${ff}-${i}.${fe}"
	fi
	curl --output rbm_pics/${l} "${f}"
	sed "sX${f}Xrbm_pics/${l}X" rbm0.html > t.html
	mv t.html rbm0.html
done

Certificate for courier esmtpd

To enable SSL/TLS support for the ESMTP, you need to have a server certificate. Usually, the installation process of the package in a Linux distribution will create a default, self-signed certificate for you. However, if you want to create a proper certificate for your site, following is some simple steps to do so.

First, you need to generate a key for your server if you don’t already have one: 

openssl genrsa -out server.key 2048

With that key, you can then generate a certificate request: 

openssl req -new -key server.key -out server.csr

If you did not customize your openssl.cnf configuration file, the above command will prompt you for the details of identify for the server in the certificate. Answer all questions as you please except for the common name “CN”, which should be the host name to connect to your server.

Now, you need to get a Certificate Authority to sign your request. For example, if you have a demoCA setup for your openssl installation, you can do: 

openssl ca -config openssl.cnf -policy policy_anything -out server.crt -infiles server.csr

This results in the certificate file server.crt. You then can combine the server key and certificate files to create the certificate file for the courier mail server. 

cat server.key server.crt > esmtpd.pem

This used to be sufficient. However, the newer version (0.73) of courier requires a “DH parameters” block in the certificate file. This can be generated and appended with: 

openssl dhparam 1024 >> esmtpd.pem

Now, you can point the “TLS_CERTFILE” in all the configuration files to the certificate esmtpd.pem and restart your server.

Example: data preprocessing with BASH

Case situation

I have run some batch jobs on a cluster to process data files for different systems (msc, ms, sh, rd) and parameters (i and w). The files are in different subdirectories:

[cjj@gust pattern]$ ls d-*/*.spd
d-msc/i275w042526.spd  d-ms/i285w025017.spd  d-rd/i295w042812.spd
d-msc/i280w040241.spd  d-ms/i290w023034.spd  d-sh/i275w051138.spd
d-msc/i285w036791.spd  d-ms/i295w020787.spd  d-sh/i280w047315.spd
d-msc/i290w031925.spd  d-rd/i270w065151.spd  d-sh/i285w043415.spd
d-msc/i295w026791.spd  d-rd/i275w060475.spd  d-sh/i290w039589.spd
d-ms/i270w034433.spd   d-rd/i280w055777.spd  d-sh/i295w035791.spd
d-ms/i275w030644.spd   d-rd/i285w051257.spd
d-ms/i280w027133.spd   d-rd/i290w046948.spd
[cjj@gust pattern]$

While, the output files are in the current directory:

[cjj@gust pattern]$ ls *.o*
i270w034433.spd.o172489  i275w060475.spd.o172496  i285w036791.spd.o172486
i270w065151.spd.o172495  i280w027133.spd.o172491  i290w023034.spd.o172493
i275w030644.spd.o172490  i280w040241.spd.o172485  i290w031925.spd.o172487
i275w042526.spd.o172484  i285w025017.spd.o172492  i295w026791.spd.o172488
[cjj@gust pattern]$

The format of the output log files are as follows:

[cjj@gust pattern]$ cat i270w034433.spd.o172489
MinTemplateNumber =  3
JT =  5
JN =  1
spikeResolution =  2
Number of initial spike patterns have been found : 562
ans = Creating surrogate data
ans = Creating time jittering surrogate data
ans = Creating neuron jittering surrogate data
Number of spike patterns have been valid by checking with sorrogate : 542
Number of spike patterns have been ruled out because of having less complex : 205
Number of valid spike patterns have been found : 337
[cjj@gust pattern]$

Problem task

Gather the stats in the log files as those marked in red.

Solution 1

This is done with a one-liner:

[cjj@gust pattern]$ for i in d-*/*.spd;do n=${i%/*};n=${n#d-};s=${i#*/};if [ -f ${s}.o* ];then w=${s%.spd};w=${w#*w}; echo ${n} ${s:1:2}.${s:3:1} $((1${w:0:2}-100)).${w:2} `grep ':' ${s}.o* | awk '{print $NF}'`;fi;done > matching_stat.txt

which can be broken down to:

for i in d-*/*.spd;do
  n=${i%/*}
  n=${n#d-}
  s=${i#*/}
  if [ -f ${s}.o* ];then
    w=${s%.spd}
    w=${w#*w}
    echo ${n} ${s:1:2}.${s:3:1} $((1${w:0:2}-100)).${w:2} `grep ':' ${s}.o* | awk '{print $NF}'`
  fi
done > matching_stat.txt

The data file generated is:

[cjj@gust pattern]$ cat matching_stat.txt 
msc 27.5 4.2526 81 75 22 53
msc 28.0 4.0241 237 217 103 114
msc 28.5 3.6791 393 371 156 215
msc 29.0 3.1925 335 322 132 190
msc 29.5 2.6791 445 437 144 293
ms 27.0 3.4433 562 542 205 337
ms 27.5 3.0644 1037 1006 331 675
ms 28.0 2.7133 1141 1093 341 752
ms 28.5 2.5017 1325 1274 462 812
ms 29.0 2.3034 1652 1609 747 862
rd 27.0 6.5151 1031 953 313 640
rd 27.5 6.0475 1042 963 345 618
[cjj@gust pattern]$