Kafka-Ops: yet another CLI utility to automate Kafka cluster management
Date August 12th, 2020 Author Vitaly Agapov
This is just an introduction of my CLI tool for idempotent management of Kafka cluster objects (topics and ACLs for now). It is already adopted by our CI/CD pipelines for preparing dedicated or multi-tenant Kafka clusters during deployment of the microservices that act as producers. So we've got rid of any manual management of Kafka topics and ACLs.
The utility is called Kafka-Ops and its source code and releases are available at https://github.com/agapoff/kafka-ops
All the needed documentation is provided in README so I won't post it here. Please refer to Github, use, feedback, contribute.
Tags: Go, Kafka
Категория:
Linux |
Нет комментариев »
Nagios style plugin for Kubernetes monitoring
Date July 4th, 2018 Author Vitaly Agapov
Usually when one thinks about the active monitoring of Kubernetes health then the Prometheus + Alertmanager comes to his mind. This is quite obvious way especially when Prometheus is already an almost mandatory component of the Kubernetes cluster. But what if I am happy with my Icinga/Zabbix/"even Nagios" setup which is monitoring my beyond-Kubernetes stuff and I want to keep everything in one place, manage the contacts, notifications as I used to? Then I need a plugin. And I have one. It is right here:
https://github.com/agapoff/check_kubernetes
The repo has a detailed README so I won't repeat it here. Just use it, feedback and contribute.
Tags: Kubernetes, Nagios
Категория:
Kubernetes |
Нет комментариев »
Yet another Fluentd deployment for Kubernetes
Date June 27th, 2018 Author Vitaly Agapov
It is always mandatory to make logs from the services available for developers and other involved persons. And this access is expected to be not too complicated, without kubectl magic and dashboards (when we are talking about Kubernetes). It is quite easy to start exporting all logs from the Kubernetes cluster to Elasticsearch. The most straightforward way is:
Tags: Docker, Elasticsearch, Fluentd, Kubernetes
Категория:
Kubernetes |
Нет комментариев »
Yet another Prometheus deployment for Kubernetes
Date June 22nd, 2018 Author Vitaly Agapov
— You don't have a right thing, friend. You've got a whole plateful of maybe a little less wrong.
I have kept silence here for some time. But at last I have something short to say. Recently I was trying to prepare gathering of metrics from bare-metal Kubernetes clusters for observing them from my favourite Grafana. And it appeared that it's quite difficult to do that using the official documentation and blog posts. Moreover the presented Grafana dashboards were not good enough for me. So when I finished my investigations and experiments and when I was finally happy with my dashboard visualizations I pushed them to Github. The repository can be found here: https://github.com/agapoff/kubernetes-prometheus. It contains the K8s manifests for deploying Prometheus with node-exporter and kube-state-metrics apps and the appropriate ConfigMap. It also contains the json files for Grafana dashboards which can be imported and used out of the box (you need only to create the Prometheus datasource).
Feel free to feedback and contribute.
Tags: Docker, Grafana, Kubernetes, Prometheus
Категория:
Kubernetes |
Нет комментариев »
Nagios style plugin for Solace Router monitoring
Date March 21st, 2017 Author Vitaly Agapov
Solace documentation contains information on some Nagios integration options and a mythical plugin for monitoring Solace from Nagios. But this plugin can not be obtained from Solace site and even from Solace support. The exact quotation from Solace support sounds as "our Nagios plugins are not a product, they are projectware that we provide as part of a paid services engagement related to integration with Nagios". Quite uncomfortable, huh.
So I have prepared own Nagios style plugin (it can obviously be used in Icinga, Shinken etc.) which covers all our needs. It makes use of SEMPv1 management protocol. This protocol is described as legacy in the official documentation but SEMPv2 is not ready yet for usage.
Tags: Nagios, Solace
Категория:
Linux, Perl |
Нет комментариев »
How to monitor Linux network quality for different directions
Date December 7th, 2016 Author Vitaly Agapov
Sometimes we might need to know if the network connection between our host and other peers is good enough. The main quality attributes of the network connection are badwidth (quite easy to explore) and the packet loss rate (quite difficult to explore). Of course we can monitor the overall TCP retransmit rate (for example using something like netstat -s | grep -i retrans), build the graph and stare at it. But we'll never know which direction is loosing the packets and where is the problem itself.
I have not found the ready-to-use solution and created my own. You can see the results right here on the screenshot.
Tags: Linux, Perl
Категория:
Linux, Perl |
Нет комментариев »
Generate RPM SPEC changelogs in Git
Date November 8th, 2016 Author Vitaly Agapov
This little note will be useful for those who is lazy enough for dealing with changelogs in RPM spec-files and lazy enough to implement own solution. I will present the script to work from inside the git repository. It can check if the changelog complies the current version and update the changelog accordingly if needed. No git tags are needed.
There are two variants of the script. One is for running on the build server (Jenkins/Bamboo/TeamCity…) right before building the RPM. And the second one is the post-commit hook. Second option can be useful if your build server has no write access to the repository or if you'd like to keep git history cleaner (this option doesn't create extra commits).
The script needs rpmspec utility to be installed. So if you are using some non-RH distro just look for the needed package. In Ubuntu you need the package named 'rpm'.
Tags: Git, Linux
Категория:
Linux |
Нет комментариев »
How to pack Java Cryptography Extension (JCE) to JDK RPM
Date July 22nd, 2016 Author Vitaly Agapov
Let's imagine you have hundreds of hosts where you automatically install or update HotSpot JDK from the RPM-repository. For example, by RPM dependencies while installing or updating some another software. You maintain your repository by yourself and you'd like to have JDK and JCE in one RPM package.
Then this tiny article will help you.
Tags: Linux
Категория:
Linux |
2 Комментариев »
How to query MS SQL Server database from Linux CLI
Date June 15th, 2016 Author Vitaly Agapov
So let's assume that we need a small keyhole to look into SQL Server database and query some data from it. Maybe we'd like to automate some reporting stuff or to make some web GUI with some information stored in MSSQL. And of course we don't want to deal with Windows.
Fortunately there is such a project as FreeTDS. It gives us a driver for using with ODBC engine. I will show how to configure it and how to retreive some data right from MSSQL.
Everything written below is true for the case of RHEL/CentOS 7.
Tags: Linux
Категория:
Linux |
Нет комментариев »
How to migrate XWIki and DokiWiki to Confluence: still easy and for free
Date May 11th, 2016 Author Vitaly Agapov
I am surely aware of the UWC project but I just wasn't satisfied with the results. So I have created my own tool to migrate articles from XWiki and Dokuwiki to Confluence. The tool is quite simple, it doesn't deal with editing history and just migrates the current information. This was my goal and it was achieved. So if you are interested in this – look into the project WikiLumper which is available on GitHub.
And if you have something to contribute then please contribute.
Tags: Atlassian, Perl
Категория:
Perl |
Нет комментариев »