After elasticsearch license changes, opensearch has forked out with Apache 2.0 opensource license in 2021. This is maintained by Amazon web services (AWS).
In this article, we will walk through building opensearch stack using opensearch, opensearch-dashboards and fluentd.
We are using debian 13 trixie, opensearch, opensearch-dashboards 3.10, fluentd 5.0.7.
Obtain opensearch, opensearch-dashboards from https://opensearch.org/downloads/ and fluentd from https://www.fluentd.org/download
Install respective packages
sudo env OPENSEARCH_INITIAL_ADMIN_PASSWORD=Set-initial-admin-password dpkg -i opensearch-3.1.0-linux-x64.deb
sudo dpkg -i opensearch-dashboards-3.1.0-linux-x64.deb fluent-package_5.0.7-1_amd64.debConfigure opensearch
Edit /etc/opensearch/opensearch.yml. Mainly you may want to tune following config
cluster.name: opensearch01
path.data: /opt/opensearch
path.logs: /var/log/opensearchConfigure opensearch-dashboards
Edit /etc/opensearch-dashboards/opensearch_dashboards.yml. Check and change following as per your environment.
server.host: 0.0.0.0
server.name: "opensearch-home"
logging.dest: /var/log/opensearch-dashboards/dashboards.log
opensearch.hosts: [https://localhost:9200]
opensearch.ssl.verificationMode: noneConfigure fluentd
Edit /etc/fluent/fluentd.conf to enable http input and output directed into opensearch database
<match mka.**>
@type opensearch
host localhost
scheme https
port 9200
ssl_verify false
logstash_format true
logstash_prefix mka
user admin
password opensearch_admin_user_password
include_tag_key true
@log_level debug
</match>
# HTTP input
# POST http://localhost:8888/<tag>?json=<json>
# POST http://localhost:8888/td.myapp.login?json={"user"%3A"me"}
# @see http://docs.fluentd.org/articles/in_http
<source>
@type http
@id input_http
port 8888
@log_level debug
</source>
Above configuration shall enable http inputs of json payload on port 8888. And insert received data (matching with mka.** tag) into opensearch instance on localhost:9200 authenticated by admin creds. In opensearch prefix shall be created as “mka-***”
Enable, start and check status of all services in following order.
Opensearch
sudo systemctl enable opensearch
sudo systemctl start opensearch
sudo systemctl status opensearchOpensearch-dashboards
sudo systemctl enable opensearch-dashboards
sudo systemctl start opensearch-dashboards
sudo systemctl status opensearch-dashboardsFluentd
sudo systemctl enable fluentd
sudo systemctl start fluentd
sudo systemctl status fluentdOpensearch has internal roles and users defined in /etc/opensearch/opensearch-security/ directory. To load and enable them run following. Later you may create own certificates and use them in opensearch.
sudo "/usr/share/opensearch/plugins/opensearch-security/tools/securityadmin.sh" -cd "/etc/opensearch/opensearch-security" -icl -key "/etc/opensearch/kirk-key.pem" -cert "/etc/opensearch/kirk.pem" -cacert "/etc/opensearch/root-ca.pem" -nhnvCheck log files of all respective components
Opensearch
sudo tail -f /var/log/opensearch/opensearch.logOpensearch-dashboards
sudo tail -f /var/log/opensearch-dashboards/dashboards.logFluentd
tail -f /var/log/fluent/fluentd.logIf all works well so far without any errors in status or logs, next add some data via fluentd http input into opensearch.
curl -X POST -H "Content-Type: application/json" -d '{"name": "manish", "city": "Madurai", "Phone": 32452342342}' http://localhost:8888/mka.loginHere mka.login in end of url is tag value that shall appear in opensearch database.
Finally visualize data in opensearch-dashboard
Connect to http://your-server-ip:5601/app/login?
Login with opensearch “admin” credentials, navigate to discover
In case you want to reset admin user password or create more internal users, edit /etc/opensearch/opensearch-security/internal_users.yml
To reset password, generate hash using /usr/share/opensearch/plugins/opensearch-security/tools/hash.sh tool. Eg:
/usr/share/opensearch/plugins/opensearch-security/tools/hash.sh -p somepassword
WARNING: nor OPENSEARCH_JAVA_HOME nor JAVA_HOME is set, will use /usr/bin/java
$2y$12$/vJEuWVpNOtUY2qPkbNMyeceaggobCkQYuUprfAuXwDnDhVf5CAwy
Now copy paste this hash password into /etc/opensearch/opensearch-security/internal_users.yml to replace existing user password and re-run
sudo "/usr/share/opensearch/plugins/opensearch-security/tools/securityadmin.sh" -cd "/etc/opensearch/opensearch-security" -icl -key "/etc/opensearch/kirk-key.pem" -cert "/etc/opensearch/kirk.pem" -cacert "/etc/opensearch/root-ca.pem" -nhnvSimilarly, to add a new user add user block in /etc/opensearch/opensearch-security/internal_users.yml like:
manish:
hash: "$2yxxxxxxxxxxxxxxxxxxxxxxxxxetLfuDTEFC"
reserved: true
backend_roles:
- "admin"
description: "Manish admin user"Add re-run securityadmin.sh as above.
List opensearch indices to locate index prefix created by fluentd
curl -k -u "admin:xxxxxx" -X GET "https://localhost:9200/_cat/indices?v"
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
green open top_queries-2025.08.14-70620 F7RozqnITUeg20oS3Y7DUA 1 0 35 4 164.2kb 164.2kb
yellow open mka-2025.08.25 7aDXpKiPTbONDHPBZxtb3w 1 1 2 0 6.1kb 6.1kb
green open .kibana_-1081426716_manish_1 WZE7vJHdR5mx0-k3UDzzJQ 1 0 2 0 12.1kb 12.1kb
green open .ql-datasources BGgZjUYaTrekvtuk_kjUuw 1 0 0 0 208b 208b
green open security-auditlog-2025.08.22 eVzHgwvsT8S4dEsxqw7pEQ 1 0 12 0 47.8kb 47.8kb
green open .kibana_92668751_admin_1 U3sF8Lc5RMSC9q8jJnqF1w 1 0 2 0 12kb 12kb
green open security-auditlog-2025.08.14 to1GKBv7QgucCLrU80E0uQ 1 0 92 0 140kb 140kb
green open security-auditlog-2025.08.25 15_zhOb0TxePMXRyQ8rNHA 1 0 36 0 337.6kb 337.6kb
yellow open mka-2025.08.21 VvBEwKnySxmF9s_Jx11pFQ 1 1 8 0 6.5kb 6.5kb
green open .opendistro_security sm4_PWBQSbS_sOUAJw0bIw 1 0 9 0 81.2kb 81.2kb
Related Posts
