Tech bytes
python based nmap port scanning
Nmap is a great tool to run scans on remote hosts and networks. python-nmap module uses underline nmap binary to run scans and adds advantage of programmable controls to get desired results. To start install python-nmap module. Here is a sample code how to use above module and run scans. Save this as scanit.py Output
Paramiko python ssh library
At times we have to use ssh connections to remote hosts/devices in programs to perform certain tasks. Here is an example that you can pick and modify to your taste.
Migrate elasticsearch data from one cluster to another
Suppose you have an existing elasticsearch cluster A and, for any reason, you want to migrate the data to cluster B.In this write up, My cluster A is on ELK 5.4 and cluster B is on ELK 6.7. This procedure is very handy to upgrade and migrate ELK on from existing to new servers. On […]
Playing with elasticsearch indices
List all indices List indices matching a pattern Delete an index Fetch available fields in an index Search something within an index List all items in an index
Syslog messages in EFK stack
You can use opensource syslog-ng tool to capture syslog messages from various sources, like remote servers, devices, local logs and so on. These logs can then be forwarded to fuentd on EFK stack. Here is a sample config of syslog-ng config to receive logs from devices and hosts on tcp and udp port 514. Received […]
EFK stack – versatile and very capable analytics platform
So far I was happily using ELK stack to feed syslog messages into Elasticsearch. In ELK stack I had used Logstash to aggregate syslogs and feed them into elasticsearch. Recently, I came across fluentd and found it quite interesting and flexible. Using fluentd with Elasticsearch and Kibana I have now build a EFK stack. EFK […]
How to interact with influxdb using REST calls
Influxdb is open source database which is predominantly used for time series data collection . I have been using influxdb on my raspberry pi to store data from IoT sensors at home. So far I was using python influxdb module to pump in data. Recently I wanted to find some clientless solution to manage data […]
Starting with Python django
Django is a open source and free python based framework for web programming. I have used python3 and django 2.1.7 on debian 9.8 in this article to write a basic django based web app. Install django Create a directory to hold are your development stuff “django-works” or whatever name you like. Create your first django […]
Publish IoT data in Influxdb and visualize in Grafana
I had build an IoT enabled weather station. Now let us build a nice platform to store time series data coming out of DTH22 (weather sensor) on NodeMCU and visualize it into Grafana. I have installed and configured influxdb and Grafana onto my raspberry pi which also hosts my MQTT broker. Weather station code transmits […]
Secure Elasticsearch and Kibana access using Apache reverse proxy
Default ELK installation may not fulfill enterprise requirements as kibana, the visualizer, works on port 5601 without any authentication and elasticsearch listens only on loopback IP on port 9200 without any authentication. You may want to run kibana on standard http/https port ie 80/443 with authentication control. Similarly, you may want to allow elasticsearch access […]