Playing with Unix "at" spool

Unix at is a handy tool to schedule one time jobs.
all u need to do is, have at installed, and run:

at now + 15 min
at> ls >/tmp/test
at> ctrl+d

this will schedule a job for u.

This will create a spooled job file at: /var/spool/at.
The list of spooled jobs looks like this:

[root@feanor at]# ls -ltr
total 128
drwx------. 2 daemon daemon    6 Oct  7  2014 spool
-rwx------  1 root   root   5593 Jun 22 07:08 a000010174f96b
-rwx------  1 root   root   5581 Jun 22 07:15 a000020174f971
-rwx------  1 root   root   5581 Jun 22 07:15 a000030174f972
-rwx------  1 root   root   5576 Jun 22 07:18 a000040174f974
-rwx------  1 root   root   5576 Jun 22 07:18 a000050174f975
-rwx------  1 root   root   5576 Jun 22 07:18 a000060174f975
-rwx------  1 root   root   5576 Jun 22 07:18 a000070174f975
-rwx------  1 root   root   5579 Jun 22 07:18 a000080174f975
-rwx------  1 root   root   5576 Jun 22 07:18 a000090174f975
-rwx------  1 root   root   5576 Jun 22 07:18 a0000a0174f975
-rwx------  1 root   root   5576 Jun 22 07:18 a0000b0174f975
-rwx------  1 root   root   5576 Jun 22 07:18 a0000c0174f975
-rwx------  1 root   root   5577 Jun 22 07:18 a0000d0174f975
-rwx------  1 root   root   5577 Jun 22 07:19 a0000e0174f975
-rwx------  1 root   root   5576 Jun 22 07:19 a0000f0174f976
-rwx------  1 root   root   5577 Jun 22 07:19 a000100174f976
[root@feanor at]# 

If you notice, the spool file name is in hex decimal.
The file name is decoded as follows:
a0-000f-0174f976
where:
- the 1stpart i am not sure what it represents, looks like it is reserved.
- the 2nd part is the job count in Hex, starting at 0x1 counting in hex until "0xf" (16) then counts to "0x10" (17) and so on.
Once the node is rebooted this count resets.
- the 3rd part is the important one, is the number of minutes since 1-1-1970 until the time of the job execution.
This is how the atd tell which job runs when.

Taking a look at the file contents, it looks like a normal shell script file with a lot of variables.
It defines, user, home shell path and lots of other variables.
last part of it is interesting part where it actually runs the job:

[root@feanor at]# tail a000100174f976
XAUTHORITY=/tmp/kde-root/xauth-0-_0; export XAUTHORITY
OLDPWD=/var/spool/at/spool; export OLDPWD
cd /var/spool/at || {
         echo 'Execution directory inaccessible' >&2
         exit 1
}
${SHELL:-/bin/sh} << 'marcinDELIMITER1d5dd9b1'
pwd

marcinDELIMITER1d5dd9b1
[root@feanor at]#

So looking at this, this job was just running a pwd commad.

So knowing this info, we can construct an at spool file similar to this one and have it run any thing for us.
To do this we can make use of the date command:

date +%s ; date +%s -d "now + 10 sec"

[root@feanor at]# date +%s ; date +%s -d "now + 10 sec"
1466595254
1466595264
[root@feanor at]#

which will return the number of seconds since 1-1-1970 and just divid this by 60 for any future time we want.

A script that uses this could look like this:

[root@feanor at]# cat create_at_spool.sh
TIME_SPEC=$1
CMD=$2

DECTIME=$(date +%s -d "${TIME_SPEC}")

TIMEPART=$(echo " obase=16; `echo "${DECTIME}/60"|bc` "|bc)
JOBPART="a001000"
echo ${JOBPART}${TIMEPART}

cp /var/spool/at/at_template.spool /var/spool/at/${JOBPART}${TIMEPART}

echo "\${SHELL:-/bin/sh} << 'marcinDELIMITER'">>/var/spool/at/${JOBPART}${TIMEPART}
echo "${CMD}" >>/var/spool/at/${JOBPART}${TIMEPART}
echo "marcinDELIMITER">>/var/spool/at/${JOBPART}${TIMEPART}
You have new mail in /var/spool/mail/root
[root@feanor at]#

One issue i noticed is that atd doesn't pick up the job from its own, it either needs to be tiggered using atd -s or another at command is run.
I will keep looking into this and update my results.

Mcollective and Regexp Host filters

Again another post just to remind me how things work with Mcollective.
We can provide the mco command with a host identity filter that would direct it to execute commands on certain hosts only.
Mcollective supports ruby regular expressions.
check them at: http://ruby-doc.org/core-2.3.1/Regexp.html

The command looks like this:

mco puppet runonce -I "/^myserv.*/"

This will match any host within the puppert resources that has a hostname starting with the string "myserv".
The key here is that when working on large scale projects this command would be able to save me a loop on all the affected nodes.

The command will of course run puppet once, but again this can be used with any mcollective plugin.

Shell script trick to sort release versions

Unix sort is not designed to sort release versions numerically.
The versions will look like:  v1.25.3 or v1.3.1.
Sort will think that 1.3 is bigger than 1.25 numerically, and thus will not sort the versions as expected.
This became an issue in a script i was creating to detect the latest snapshot version on an apache archiva repo to support release work.

To work around this, i have replace the "." with "," before i did the sort and then replaced the "." back.
This will make sort think we are looking at different numbers and the "." will not interfere, thus will look at the versions as: v1,25,3 and v1,3,1 and thus will think that 25 is bigger than 3 as we expect.

Below is the script:

cat get_snapshots.sh
for SERVICE in `grep fileSourceURL archiva_URL_list |sort |cut -d":" -f3,6-|cut -d"/" -f1-9|tr -d "'"|tr -d " "|sed -e "s/$/\//g"`
do
    serv_name=`echo ${SERVICE}|cut -d":" -f1`
    url=`echo ${SERVICE}|cut -d":" -f2-`
    SNAPSHOT=`curl ${url} 2>/dev/null|grep '<li>'|egrep -v "xml|proj|maven|2016|2015"|cut -d">" -f3-|cut -d"<" -f1|tr "." ","|sort -n -t"," -k1,2|tr "," "."|tail  -1`
    echo ${serv_name}:${SNAPSHOT}
done

The curl command is piped to an egrep that would remove all the unwanted HTML tags and other maven generated files part of the build.

As can be seen the sort is between 2 tr command, first will replace the "." with a "," and the 2nd will revert the "." back.
Sort uses the "," as separator and it does the sort on the first field then on the 2nd.
I am assuming minor versions are not updated as often but this script can still be extended further to even longer version numbers.

Could be useful some day !!

A simple (web)socket server with python

Below is a simple socket or u can say websocket server in python.
Used this to test apache websocket proxy for one project.
Though to textend it a bit so i can play with it using telnet. 
The server uses python threads which is a good place to learn how python threads work when tied to sockets.
The thread runs the handle function and passes client connection socket returned by accept.
Also needed to do some string operations to make the message going to the telnet client look good.
Will be extending this more and will try to link it to a JS front end and see how can i make use of that.

Python is really good :)


[root@Vardamir ~]# cat pyws.py
#!/usr/bin/env python

import socket, threading, time

def handle(s):
  print "new connction"
  print "connection detailes:"
  print repr(s.recv(4096))
  s.send('''
HTTP/1.1 101 Web Socket Protocol Handshake\r
Upgrade: WebSocket\r
Connection: Upgrade\r
WebSocket-Origin: http://localhost:9999\r
WebSocket-Location: ws://localhost:9999/\r
WebSocket-Protocol: sample
  '''.strip() + '\r\n\r\n')
  time.sleep(1)
  s.send('\x00hello_world\xff')
  time.sleep(1)
  s.send('\r\n')
  time.sleep(1)
  s.send('This is a test websocket server in python')
  s.send('\r\n')
  time.sleep(1)
  s.send('\r\n')
 
  while 1:
        data=str(repr(s.recv(4096)))
        data=data[1:-5]
        print data
        time.sleep(1)
        s.send("hello i got this "+data+" from u :)")
        s.send('\r\n')
        s.close()
        break


s = socket.socket()
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind(('', 9999));
s.listen(1);
print "Simple Server started on port 9999"
while 1:
  clientconn , clientaddr = s.accept();
  clientThread = threading.Thread(target=handle, args=(clientconn, ))
  clientThread.start()