Spoofing user agent string and X-Forwarded-For header with CRUL

curl is a handy tool used to test web apps from unix command line.
below are some hints to use it for spoofing ips and user agent strings to allow testing from command line.

Most modern websites block the defaul curl user agent of fear of hacks and DDoS attackers.

to over come this, you would need to use curl -A and set the user agent string to a known browser string.
the strings can be found here: http://www.useragentstring.com/pages/useragentstring.php

to further spoof ur IP, we can make use of X-Forwarded-For header this will fool the site to think ur coming from this IP, this just works for HTTP, no actual IP spoofing is there.

To do this from curl, we use curl -H.
the full command looks like this:

 curl -v https://www.msn.com -k -A "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Firefox/45.0" -k -H "X-Forwarded-For: 163.121.63.241"

This comes in handy to test blocked IP ranges and user agents.

Black tooltips for Eclipse

This is to fix the black tooltip issue with eclipse :)

turned out to be a linux KDE issue in my case:
http://askubuntu.com/questions/45001/how-to-fix-black-tooltips-in-eclipse
 

InfluxDB+Grafana

This post is to describe how to use influxdb and Grafana to plot system metrics and any other metrics that can use a time series.

First we need to install both influxdb and Grafana on Centos 7.
http://grafana.org/download/
https://influxdata.com/downloads/

Both packages are standalone and doesn't any dependencies (which is cool compared to the pain of installing graphite !!)

Once installed  you need to start both tools with systemctl and once up will be accessible from web locally as:
http://localhost:8083/ (Influxdb)
http://localhost:3000/ (Grafana)

influxdb config file /etc/opt/influxdb/influxdb.conf
i didn't change any thing in this as i intended to use influx command line interface to fill in data.

coming to filling data, we need to create a new DB and use it:

[root@localhost ~]# influx
Connected to http://localhost:8086 version 0.9.4.2
InfluxDB shell 0.9.4.2
> create database test
> show databases
name: databases
---------------
name
_internal
mydb
localhost
test

> 
 


Once the DB is created, we can use the below sample script to fill in data using command line:

[root@localhost ~]# cat cpu_collect.sh
CPU_SNAP=`top -b -n2 -d0.05 |grep "%Cpu(s):" |tail -1|tr -s " "|cut -d" " -f2,4,6,8,10`

usr_cpu=`echo $CPU_SNAP |cut -d " " -f1`
sys_cpu=`echo $CPU_SNAP |cut -d " " -f2`
nic_cpu=`echo $CPU_SNAP |cut -d " " -f3`
idl_cpu=`echo $CPU_SNAP |cut -d " " -f4`
wat_cpu=`echo $CPU_SNAP |cut -d " " -f5`


echo "usr_cpu=${usr_cpu} sys_cpu=${sys_cpu} nic_cpu=${nic_cpu} idl_cpu=${idl_cpu} wat_cpu=${wat_cpu}"
influx <<EOF
        use localhost
        insert  cpu usr_cpu=${usr_cpu},sys_cpu=${sys_cpu},nic_cpu=${nic_cpu},idl_cpu=${idl_cpu},wat_cpu=${wat_cpu}
EOF
[root@localhost ~]#

We need to put this in cron to run every 2 minutes so it fills the influxdb with data.
this will create a metric (table) called cpu with cpu data shown as below:








this will then be used by grafana to plot the time series data.
switching to grafana, login using admin/admin.
once logged in you will see the dashboard screen :



now we need to create a data source so that grafana is able to connect to influxdb.
use the info similar to below screen shot:



Once done, go back to the home screen
on the banner, select the dashboards drop-down and click new to create a new dashboard:

 

once created, the new dashboard will contain one raw by default,  the raw is the container for graphs.
to create a new graph, hover your mouse over the green rectangle at the side it will pub-up and once u click you will see a menu as below:




once a graph is created, we need to select the proper data source from the drop-down menu similar to below screen shot:


once selected, we need to click the +Query button, this will bring query builder that will bring our metric from influxdb to be graphed.



The in the query builder we see a couple of queries that can be used, lets edit the first one, in the from field, start typing the metric name; cpu in our case, you grafana will auto-complete for you.
also in the select box, click the field(value) this will bring a drop-down list of all valid values in the cpu metric, check the below screen shot:



lets remove the 'B' query and click save in the top tool bar.
then lets go to the dashboards once more and select our new dashboard, now you will see the graph of the cpu metric - idle cpu percentage value:



Grafana is very rich dashboarding tool, you can now explore the whole thing and check how u can make use of it.

enjoy :)