Then, we used the perf_data_converter tool to convert perf file perf.data to profile.proto and used pprof to analyze the results. This gave us a rough idea of functions taking up most of the CPU time. To take a closer look, we started doing some profiling upon the StatsD exporter and used the perf tool to sample stacks. When the client request rate climbed up to several thousand requests per second, we spotted a high CPU usage of the StatsD exporter that took one and a half cores on an AWS m4.large instance. On Kong Cloud, various StatsD events are generated for each request. In the example below, two StatsD events are translated according to the mapping config to the left. The exporter has a mapping config that maps the StatsD metric to a Prometheus metric. The StatsD Prometheus exporter is a daemon that listens for StatsD events and exports them in Prometheus exposition formats. On Kong Cloud, we use the StatsD Prometheus exporter in our metrics pipeline to measure KPIs (Key Performance Indicator) of our service.
The above example represents a metric called with the type of counter and the value of 123. A typical StatsD event looks like:Įvery StatsD event is a string in a format of :|. StatsD is a metrics server that accepts events from UDP or TCP protocol and export them to various backends. In this blog post we discuss the use case of StatsD and Prometheus on Kong Cloud, the performance problem we found, and the way we proposed to solve it. That may be the reason why I can not query the custom metrics which is in statsd exporter from Prometheus.įollowing is the configuration in Prometheus.Kong Cloud has been using StatsD and Prometheus heavily in monitoring and metrics collecting. Prometheus and Statsd exporter are not talking to each other. I can see that Prometheus is set up at But In the Status->Targets section I do not see Statsd Exporter.K6 exits after the script execution is done. In the docker desktop that all the four services are up and running.
When I use the command docker compose up I can see I tried giving the ip address, localhost too.) scrape_configs:įollowing is the mapping file for statds exporter for K6 mapping. (I don't see this job listed in the Prometheus. loadtesting/prometheus.yml:/etc/prometheus/prometheus.ymlįollowing is my Prometheus yaml file which I believe is trying to scrape the data from the same port where statds exporter is outputting results. grafana/datasources:/etc/grafana/datasources/
GF_INSTALL_PLUGINS=michaeldmoore-scatter-panel loadtesting/statsd_mapping.yml:/tmp/statsd_mapping.ymlĬommand: The prometheus job is not getting listed in the Status->Targets page on prometheus.įollowing is my docker-compose file. Grafana - which will connect to Prometheus and show visualization.īut what happens is that Statsd metrics are generated including my custom metrics.Prometheus which will scrape the results from statsd exporter.
STATSD EXPORTER FREE
If anyone has a simpler solution than this please feel free to suggest.). Send the result to Statsd Exporter (The idea is to use Prometheus as the database but since K6 doesn't support directly support Prometheus, I am using Statsd.
0 kommentar(er)
