ElasticSearch Commands to get you into using it.

Personally, I run Elasticsearch, Kibana, Metricbeat, and Filebeat in a single docker-compose stack managed with Portainer. Kibana is useful for viewing data, although I don't like that it doesn't let you edit data. I connect TubeArchivist using the elastic password generated by the compose stack. I also want to use Elasticsearch for other purposes and avoid running a separate instance.

From kibana i just created an api key for ta_channels to update data within them. Here's a curl command to generate an api key without kibana below.

Create API key scoped to specific indices (HTTP)

curl -s -u 'elastic:ELASTIC_PASS' \
  -H 'Content-Type: application/json' \
  -X POST 'http://localhost:9200/_security/api_key' \
  -d '{
    "name": "ta_scoped_key",
    "expiration": "30d",
    "role_descriptors": {
      "ta_scoped_role": {
        "cluster": ["monitor"],
        "index": [
          { "names": ["ta_channels_*"], "privileges": ["read","write"] },
          { "names": ["ta_metadata"],    "privileges": ["read","write","create_index"] }
        ]
      }
    }
  }'

HTTPS (with CA)

curl -s --cacert /path/to/chain.pem -u 'elastic:ELASTIC_PASS' \
  -H 'Content-Type: application/json' \
  -X POST 'https://localhost:9200/_security/api_key' \
  -d '{"name":"ta_scoped_key","expiration":"30d","role_descriptors":{"ta_scoped_role":{"cluster":["monitor"],"index":[{"names":["ta_channels_*"],"privileges":["read","write"]}]}}}'

Save the JSON response (it contains id and api_key), then build the ApiKey header:

Looks like this:

{"id":"F0eWBJ0BLX_vEATxQJuu","name":"ta_scoped_key","expiration":1763932732593,"api_key":"39RandomLettersandNumbers","encoded":"60RandomNumbersandLettsasldkfjwithA=="}

Use the 'encoded' key and not the 'api_key'. Not sure why, but that's what I had to use to work.

Test using the API key

curl -s -H "Authorization: ApiKey $AUTH" http://localhost:9200/_security/_authenticate

Creating Another User

curl -u 'elastic:Yourhardrandompassword' \
  -X POST "http://localhost:9200/_security/user/sickprodigy" \
  -H 'Content-Type: application/json' \
  -d '{"password":"PasswordforUser","roles":["my_readonly_role"],"full_name":"Sick Prodigy","email":"sick@sickgaming.net"}'

Creating another user with full Privs (SuperUser)

I prefer to have a user with full privs other than elastic, although TubeArchivist apparently uses elastic(default superuser)

curl -u 'elastic:Yourhardrandompassword' -X POST "http://localhost:9200/_security/user/sickprodigy" \
  -H 'Content-Type: application/json' \
  -d '{
    "password": "SomeHardPassword",
    "roles": ["superuser"],
    "full_name": "SickProdigy",
    "email": "sickprodigy@sickgaming.net"
  }'

Query certain channel within ta_channel:

The channel ID can be found on TubeArchivist, got to channel and in URL "https://tubearchivist.rcs1.top/channel/UChOve2dsTRMrW8DslLKJ9eg" after channel/ is channel ID. You can test around with query and see what comes back, but this usually bring back the exact channel you want.

curl -X POST "http://es:9200/ta_channel/_search?pretty" \
  -H "Authorization: ApiKey "YourRandomAPIkey123455123123=="" \
  -H "Content-Type: application/json" \
  -d'
{
  "query": {
    "query_string": {
      "query": "Channel ID"
    }
  }
}'

You should get back an example like this: Example-channel-info-elasticsearch.json

I'm using this as an example to update channels in TubeArchivist that are missing data. I wish kibana would let me do it. Maybe I can Just haven't figured it out yet.

3.6 KiB Raw Blame History