Tech

Storage#

Hkube support built-in storage options:

• S3 - Amazon object storage (default)
• FS - Files system
• Redis - in-memory data structure store.
• Etcd - key value store.
• Custom - write your own, PR are welcomed

Logging#

When your algorithm write logs to the stdout/stderr, Hkube will automatically collect these logs, add metadata and write these logs in a JSON format. You don't need to implement any special logger in your algorithm. If you want to write custom JSON with your specific fields....

{
  "stream": "stdout",
  "log": "This is to let you now we are connected",
  "type": "connected",
  "extra": {
    "foo": "bar"
  }
}

The cluster-level logging is outside the scope of Hkube. However, you can implement Logging Using Elasticsearch and Kibana. Hkube add metadata to logs, so you can filter the field meta.internal.algorithmName.

You can import our Kibana dashboard in this link.

Tracing#

Hkube push by default traces to Jaeger, using the opentracing. these traces are used by our dashboard

Monitoring#

Hkube collect internal metrics, if prometheus is installed in your cluster, you can use grafana to monitor your algorithms in the cluster.

You can import our Grafana dashboard in this link.