# Set up the Teleport Event Handler Plugin with the Teleport Kubernetes Operator

In this guide, we will set up the Teleport Event Handler plugin with credentials to authenticate to the Teleport cluster and access the events API.

We will deploy the Teleport Event Handler plugin using the `teleport-plugin-event-handler` Helm chart and configure it to use the Teleport Kubernetes Operator to generate the credentials.

If your Teleport cluster does not have the Teleport Kubernetes Operator deployed, follow the [guide to set up the Teleport Kubernetes Operator ](https://goteleport.com/docs/zero-trust-access/infrastructure-as-code/teleport-operator.md). If you wish to deploy the Teleport Event Handler Plugin in Kubernetes without the Teleport Kubernetes Operator, see [Set up the Event Handler with tctl](https://goteleport.com/docs/zero-trust-access/export-audit-events/event-handler-setup.md).

## How it works

The Event Handler plugin is a binary that runs independently of your Teleport cluster. It authenticates to your Teleport cluster using mutual TLS to begin forwarding events.

The Teleport Kubernetes Operator is a Teleport client that you install using the `teleport-operator` Helm chart. It is responsible for managing the Teleport resources to set up Machine & Workload Identity, `tbot`, for the `teleport-plugin-event-handler` Helm chart. This will generate the credentials to establish the mutual TLS connection between the Event Handler and the Teleport cluster.

## Prerequisites

- A running Teleport cluster. If you want to get started with Teleport, [sign up](https://goteleport.com/signup) for a free trial or [set up a demo environment](https://goteleport.com/docs/get-started/deploy-community.md).

- Docker version v20.10.7.

- A Kubernetes cluster to run the Teleport Event Handler plugin.

- [Teleport Kubernetes Operator](https://goteleport.com/docs/zero-trust-access/infrastructure-as-code/teleport-operator.md) running in the Kubernetes cluster where you will install the plugin.

- [Helm](https://helm.sh/docs/intro/quickstart/)

- [kubectl](https://kubernetes.io/docs/tasks/tools/)

- On your workstation, create a folder called `event-handler`, to hold configuration files and plugin state:

  ```
  $ mkdir -p event-handler
  $ cd event-handler
  ```

## Step 1/2. Install the Event Handler plugin

To allow Helm to install charts that are hosted in the Teleport Helm repository, use `helm repo add`:

```
$ helm repo add teleport https://charts.releases.teleport.dev
```

To update the cache of charts from the remote repository, run `helm repo update`:

```
$ helm repo update
```

## Step 2/2. Set up the Event Handler plugin

In this section, you will set up the Teleport Event Handler plugin and generate credentials that the plugin will use for authentication.

### Generate a starter config file

Generate a configuration file with placeholder values for the Teleport Event Handler plugin.

Run the `configure` command to generate a sample configuration. Assign teleport.example.com:443 to the DNS name and port of your Teleport Auth Service or Proxy Service. Assign fluentd.fluentd.svc.cluster.local to the DNS name of your log forwarder.

```
$ docker run -v `pwd`:/opt/teleport-plugin -w /opt/teleport-plugin public.ecr.aws/gravitational/teleport-plugin-event-handler:18.7.3 configure . teleport.example.com:443 --dns-names=fluentd.fluentd.svc.cluster.local
```

In order to export audit events, you'll need to have the root certificate and the client credentials available as a secret. Use the following command to create that secret in Kubernetes:

```
$ kubectl create secret generic teleport-event-handler-client-tls --from-file=ca.crt=ca.crt,client.crt=client.crt,client.key=client.key
```

This will pack the content of `ca.crt`, `client.crt`, and `client.key` into the secret so the Helm chart can mount them to their appropriate path.

The plugin generates several setup files:

| File(s)                            | Purpose                                                             |
| ---------------------------------- | ------------------------------------------------------------------- |
| `ca.crt` and `ca.key`              | Self-signed CA certificate and private key for Fluentd              |
| `server.crt` and `server.key`      | Fluentd server certificate and key                                  |
| `client.crt` and `client.key`      | Fluentd client certificate and key, all signed by the generated CA  |
| `teleport-event-handler-role.yaml` | `user` and `role` resource definitions for Teleport's event handler |
| `teleport-event-handler.toml`      | Example event handler configuration                                 |
| `fluent.conf`                      | Fluentd plugin configuration                                        |

Running the Event Handler separately from the log forwarder

This guide assumes that you are running the Event Handler on the same host or Kubernetes pod as your log forwarder. If you are not, you will need to instruct the Event Handler to generate mTLS certificates for subjects besides `localhost`. To do this, use the `--dns-names` flag of the `teleport-event-handler` configure command.

For example, if your log forwarder is addressable at `fluentd.example.com`, you would run the following `configure` command:

```
$ teleport-event-handler configure --dns-names=fluentd.example.com
```

The `--dns-names` flag accepts a comma-separated list of DNS names. It will append subject alternative names (SANs) to the server certificate (the one you will provide to your log forwarder) for each DNS name in the list.

If you have an existing Fluentd setup with TLS, issue a client certificate and key from the same certificate authority for the Teleport Event Handler to use.

### Configure tbot for the Event Handler plugin

In the generated configuration file `teleport-plugin-event-handler-values.yaml`, replace the `teleport` section with the `crd` and `tbot` sections. Assign operator-namespace to the namespace where the Teleport Kubernetes Operator is running in. Assign teleport.example.com:443 to the DNS name and port of your Teleport Proxy Service:

```
# values.yaml
crd:
  create: true
  namespace: operator-namespace

tbot:
  enabled: true
  clusterName: teleport.example.com
  teleportProxyAddress: teleport.example.com:443

```

---

WARNING

Ensure the `teleport` section is removed, since `tbot` is performing the authentication to the cluster.

---

This will generate the Event Handler role and bot user for the Machine & Workload Identity bot, `tbot`, to authenticate as when connecting to your Teleport cluster.

### Specify a join token for tbot

The `tbot` deployment authenticates to the Teleport cluster using a join token created by the `crd` section. Depending on where you deploy the Event Handler plugin relative to your Teleport cluster, you may need to modify the join token specification for `tbot` to properly authenticate.

**Same Kubernetes cluster as Teleport cluster**

When running `teleport-plugin-event-handler` in the same Kubernetes cluster as the Teleport cluster (eg. via the `teleport-cluster` Helm chart), we can use the default join token specification: `kubernetes` in\_cluster join method.

**External Teleport cluster**

When running `teleport-plugin-event-handler` against an external Teleport cluster not in Kubernetes, you must modify the join token specification using `crd.tokenSpecOverride`. We recommend either the `kubernetes` JWKS join method or the `kubernetes` OIDC join method.

**Kubernetes JWKS**

```
# values.yaml
crd:
  create: true
  namespace: operator-namespace
  tokenSpecOverride:
    join_method: kubernetes
    kubernetes:
      type: static_jwks
      static_jwks:
        jwks: |
          # Place the kubernetes JWKS here (`kubectl get --raw /openid/v1/jwks`)
          {"keys":[--snip--]}

```

**Kubernetes OIDC**

```
# values.yaml
crd:
  create: true
  namespace: operator-namespace
  tokenSpecOverride:
    join_method: kubernetes
    kubernetes:
      type: oidc
      oidc:
        # Insert the OIDC issuer URL here, it will vary depending on your
        # cluster and cloud provider.
        issuer: https://oidc.eks.us-west-2.amazonaws.com/id/my-cluster

```

### Example configuration file

Your helm configuration file `teleport-plugin-event-handler-values.yaml` should now contain settings similar to the following. We will configure the `fluentd` section to your log management solution of choice in the following guide.

```
eventHandler:
  storagePath: "./storage"
  timeout: "10s"
  batch: 20
  # concurrency is the number of concurrent sessions to process. By default, this is set to 5.
  concurrency: 5
  # The window size configures the duration of the time window for the event handler
  # to request events from Teleport. By default, this is set to 24 hours.
  # Reduce the window size if the events backend cannot manage the event volume
  # for the default window size.
  # The window size should be specified as a duration string, parsed by Go's time.ParseDuration.
  windowSize: "24h"
  # types is a list of event types to search when forwarding audit
  # events. For example, to limit forwarded events to user logins
  # and new Access Requests, you can assign this field to:
  # ["user.login", "access_request.create"]
  types: []
  # skipEventTypes lists types of audit events to skip. For example, to forward all
  # audit events except for new app deletion events, you can assign this to:
  # ["app.delete"]
  skipEventTypes: []
  # skipSessionTypes lists types of session recording events to skip. For example,
  # to forward all session events except for malformed SQL packet events,
  # you can assign this to:
  # ["db.session.malformed_packet"]
  skipSessionTypes: []

crd:
  create: true
  namespace: operator-namespace

tbot:
  enabled: true
  clusterName: teleport.example.com
  teleportProxyAddress: teleport.example.com:443

fluentd:
  url: "https://fluentd.fluentd.svc.cluster.local/events.log"
  sessionUrl: "https://fluentd.fluentd.svc.cluster.local/session.log"
  certificate:
    secretName: "teleport-event-handler-client-tls"
    caPath: "ca.crt"
    certPath: "client.crt"
    keyPath: "client.key"

persistentVolumeClaim:
  enabled: true

```

Consult the [Helm chart reference guide](https://goteleport.com/docs/reference/helm-reference/teleport-plugin-event-handler.md) for more information on the `crd` and `tbot` sections.

## Next steps

You have now configured the Teleport Event Handler plugin with credentials to access the Teleport events API.

If you are new to exporting audit events with Teleport, read [Forwarding Events with Fluentd](https://goteleport.com/docs/zero-trust-access/export-audit-events/fluentd.md) to learn the basics of how our Event Handler plugin works. While this guide focuses on Fluentd, the Event Handler plugin can export audit events to any endpoint that ingests JSON messages via HTTP.

Next, read our guides to setting up the Event Handler plugin to export audit events to your solution of choice:

- [Export Teleport Audit Events with the Elastic Stack](https://goteleport.com/docs/zero-trust-access/export-audit-events/elastic-stack.md): How to configure the Event Handler plugin to forward Teleport audit logs to Logstash for ingestion in Elasticsearch so you can explore them in Kibana.
- [Export Teleport Audit Events with Panther](https://goteleport.com/docs/zero-trust-access/export-audit-events/panther.md): How to configure the Event Handler plugin to send logs to Panther via Fluentd so you can explore your audit events in Panther.
- [Export Teleport Audit Events with Splunk](https://goteleport.com/docs/zero-trust-access/export-audit-events/splunk.md): How to configure the Event Handler plugin to send logs to Splunk's Universal Forwarder so you can explore your audit events in Splunk.
- [Export Teleport Audit Events with Datadog](https://goteleport.com/docs/zero-trust-access/export-audit-events/datadog.md): How to configure the Event Handler plugin to export audit logs to Datadog via Fluentd.