Apache: How to conditionally inject Authorization header

 In one of the cases I was helping with, I ran into a requirement that needed to have a webhook be sent from Jira to Jenkins to trigger a build based on certain condition.
You can trigger a webhook from Jira based on a Jira search query or on a certain action, eg: upon creation of an issue in a project for example.

The problem is that Jira doesn't support using userinfo fields in the URL, which is actually deprecated and should be used as per the RFC https://tools.ietf.org/html/rfc3986, and thus, Jira would have a way to send authentication information out of the box to Jenkins to trigger a build.

To solve this, I proposed to run Jenkins behind an Apache reverse proxy and use a virtual host definition similar to the below:

<VirtualHost *:80>
     ServerName feanor
     DocumentRoot /var/www/html/

     SetEnvIf Remote_Addr "10.0.0.12" buildtriggerjira
     RequestHeader set Authorization "Basic c2hlcmlmOnNoZXJpZg=="   env=buildtriggerjira

     ProxyPreserveHost on
     ProxyVia on
     ProxyPass "/jenkins" "http://feanor:8080/jenkins"
     ProxyPassReverse "/jenkins" "http://feanor:8080/jenkins"
</VirtualHost>

The above configuration will inject an Authorization header in the incoming request on condition that the remote address is the IP address of the Jira server.
This works by setting an Apache environment variable to a certain value in the SetEnvIf condition and then setting the Authorization header if the variable has that value set.

More checks can be made to harden the condition by checking more request fields like the user-agent string or other headers set by the Jira http client sending the webhook request.

Parallel execution in shell script using GNU parallel

Occasionally, one needs to look for certain strings inside very large files.
Very large file here is a text file that is more than 1GB in size.
The file could be opened by Linux tools such as less, but it becomes challenging to navigate the file and search inside it.
Also if we have a much bigger file, things can become more complex to handle with a single threaded application like less.
Things can become even more challenging if we have multiple files of that size, then one needs to either use grep and sed to look for patterns and do edits without the need to load the file(s) in memory.
Even with those command line filters, which mainly act on the input stream, doing a casual grep can take time to provide results.

To solve this, we would need to use the GNU parallel command to make use of CPU parallel execution.
parallel allows commands to be run in separate processes either on the same host or on multiple hosts, that can run in parallel processes, those would then do processing for parts of the input. parallel then collects the output of those processes and generates the result of the execution jobs.

A simple example for running parallel is below:

sherif@ulmo ~/logs $ < catalina.out parallel --pipe grep -C 4 'OutOfMemory'

The above command pipes the catalina.out that is redirected to standard input to the grep -C to look for the string.
The above form uses defaults block size (1MB) and the default number of jobs will be created (same as number of CPU cores available corresponding to 100%).

Using more optimization for the parallel command, we can achieve more efficiency compared to running a single thread filter, below is a comparison done for execution time for a standard egrep done on a 2GB file and same using parallel with 8 jobs and 100MB block size: 

[root@feanor ~]# time egrep -ni "^/ditto" all_files.txt |wc -l
82

real    0m5.401s
user    0m4.773s
sys     0m0.210s


[root@feanor ~]# time parallel --pipepart --joblog joblog1 -a all_files.txt --block 100M -P8 -k egrep -ni "^/ditto" |wc -l
Academic tradition requires you to cite works you base your article on.
When using programs that use GNU Parallel to process data for publication
please cite:

  O. Tange (2011): GNU Parallel - The Command-Line Power Tool,
  ;login: The USENIX Magazine, February 2011:42-47.

This helps funding further development; AND IT WON'T COST YOU A CENT.
If you pay 10000 EUR you should feel free to use GNU Parallel without citing.

To silence the citation notice: run 'parallel --bibtex'.

82

real    0m2.819s
user    0m7.224s
sys     0m1.801s
[root@feanor ~]#

As you can see, parallel was able to produce the same result in half the time taken by a single threaded egrep.
The fact that the user time is much higher for execution done by parallel, means that parallel made more use of CPU cores as the total CPU user time is higher than actual time (real) take by the process.
(for more information about the time command check this stackoverflow thread: https://unix.stackexchange.com/questions/40694/why-real-time-can-be-lower-than-user-time)

For more information and examples for using parallel, please check the excellent GNU parallel man page: https://www.gnu.org/software/parallel/man.html

Usign Java Keytool to create a selfsigned SSL Certificate with SAN extentions

Recently I was doing some testing on Java Identity and Access management on premise tools and SAML integration with Apache mod-auth-mellon and other Java applications supporting SAML 2.0.
I needed to create a selfsigned SSL cert for a Java application so that I can ensure the whole SAML traffic is encrypted and secure.
Since I wanted to test multiple apps using multiple hostnames on the same server, I needed to have a generic certificate to save time.
Since I am also not a big fan of wildcard certs, and those also have the limitation of working only under a single top-level domain, I elected to use the Subject Alternative Name SSL certificate extension (SAN); so that I can define multiple definite none wildcard names and I can also have multiple domains and IP addresses if needed.

Creating a SAN SSL cert using keytool turned out to be straight forward, we just need to add the keytool option "-ext" and use the SAN name argument and then list all the SAN types needed.
In below example we use the DNS type and the IP type:

sherif@Luthien:~$ keytool -genkey -keyalg RSA -alias selfsigned -keystore keystore.jks -storepass password -validity 730 -keysize 2048 -ext san=dns:localhost,dns:localhost.com,dns:luthien,dns:redash.luthien.tst,dns:luthein.tst,ip:192.168.56.101

 The resulting certificate would look like below:

sherif@Luthien:~$ keytool -list -v -keystore keystore.jks
Enter keystore password: 
Keystore type: jks
Keystore provider: SUN

Your keystore contains 1 entry

Alias name: selfsigned
Creation date: Feb 8, 2020
Entry type: PrivateKeyEntry
Certificate chain length: 1
Certificate[1]:
Owner: CN=redash.luthien.tst, OU=Sherif Redash test, O=Sherif Tests, L=Cairo, ST=Cairo, C=EG
Issuer: CN=redash.luthien.tst, OU=Sherif Redash test, O=Sherif Tests, L=Cairo, ST=Cairo, C=EG
Serial number: 552e3728
Valid from: Sat Feb 08 14:48:35 CET 2020 until: Mon Feb 07 14:48:35 CET 2022
Certificate fingerprints:
     MD5:  C9:B6:DF:E7:3A:5E:EB:6C:97:C7:CD:8F:99:5A:9A:CD
     SHA1: EE:B7:C6:6F:C5:9C:15:36:5F:A7:12:95:4E:AA:8B:59:FB:08:B6:17
     SHA256: 25:3A:07:8D:FE:5B:FF:74:BE:E7:5F:EA:51:5E:8A:D8:31:41:7E:39:33:36:95:F0:27:C1:D8:2B:CE:54:FB:DA
Signature algorithm name: SHA256withRSA
Subject Public Key Algorithm: 2048-bit RSA key
Version: 3

Extensions:

#1: ObjectId: 2.5.29.17 Criticality=false
SubjectAlternativeName [
  DNSName: localhost
  DNSName: localhost.com
  DNSName: luthien
  DNSName: redash.luthien.tst
  DNSName: luthein.tst
  IPAddress: 192.168.56.101
]

#2: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: F5 18 B2 15 5F 3F B9 63   E3 0F D3 8C 58 5A A0 67  ...._?.c....XZ.g
0010: C7 84 EF E1                                        ....
]
]

*******************************************
******************************************

As you can see, we have created the certificate with the SAN extensions that would allow us to use multiple names.

Please take a look at the Java keytool official documentation from Oracle: https://docs.oracle.com/javase/8/docs/technotes/tools/unix/keytool.html