Receive and process logs using Fluentd and Python
In normal use cases, we just have to receive logs from multiple hosts or devices, or have to forward logs from one syslog collector to another to further store them in filesystem or elastic like databases. In this case, I will discuss a scenario, where the ask is to filter logs by looking at the […]
RSA PKI based encryption and decryption in python
Generate RSA key pair in pem format Remember above pem file contains private and public key. Keep it secure with yourself. Extract public key out of pem file Now you have private key embedded in private.pem file along with public key. And a copy of public key is available in public.pem file. You can share […]
Enterprise grade Json Schema validator
In the world of enterprise grade microservices, data payload validation between webservices must be centrally controlled. For eg, in a eco system there are multiple webservices, say ws1, ws2, ws3. If ws1 gets json input data from some web form, ws1 processes it and passes its output json payloads to ws2, ws3 as input. Now […]
GPS data out of NEO M8N using python
I was thinking of building own low stratum NTP server for home. Stratum 0 NTP services are available out of atomic clocks, satellites or via some radio links. Any GPS module can easily get datetime (UTC) from satellite transmissions along with latitude, longitude (for geo positioning), speed, altitude etc. To build the project, I bought […]
Raspberry pi zero w based car dashcam
My pi zero board was lying unused since last couple of year. So I thought to make some use of it and decided to build a car dashcam. What all you need ? Raspberry pi zero w board, microSD memory card (>8GB), pi camera I have used raspbian lite OS and python3 to develop scripts. […]
Python remote ssh using public key
Quite often we come across scenarios where putting passwords in config or code is not allowed. In such situations, we can still do remote ssh using netmiko module combined with public key based authentication. Here is the sample code:
Remote syslog using python
This is quite handy way of sending informative or debug level message to a remote syslog server. And it is quite easy. Here is the basic code
Starting with Mongodb
Installation and configuation Download following mongodb packages from https://repo.mongodb.org/yum/redhat/ Install packages: Edit /etc/mongod.conf and amend systemLog->path to desired log location and storage->dbPath to desired database location. For eg, if storage->dbPath is set to /mongodb/data, then also change owner of this directory as following: Start mongodb service and tail logs in parallel View logs: Once package […]
Python pandas dataframe to Influxdb with column and other tags
Python pandas is very powerful data processing library. It can take large data from variety of sources (CSVs, databases, excel etc) and process it in memory. After processing the data you may have to dump it in influxdb. In influxdb each measurement has tags and fields. While writing pandas dataframe to influxdb we can define […]
Nested variables in Grafana
Quite often you want to use one variable of Grafana as input in query to derive next variable value set. Think of a typical example, you have 10 host machines and each host has several network interfaces. And you want to display stats of each interface of selected host. Now, when you select “host” variable […]