Doc: Add tutorial for filter-elastic_integration

docs/static/ea-integration-tutorial.asciidoc

@@ -0,0 +1,185 @@
+[[ea-integrations-tutorial]]
+=== Tutorial: {ls} `elastic_integration filter` to extend Elastic {integrations}
+++++
+<titleabbrev>Tutorial: {ls} `elastic_integration filter`</titleabbrev>
+++++
+
+You can use {ls} to transform events collected by {agent} using our {integrations-docs}[Elastic integrations]. 
+You get the benefits of Elastic integrations--such as the simplicity of ingesting data from a wide variety of data 
+sources and ensuring compliance with the {ecs-ref}/index.html[Elastic Common Schema (ECS)]--combined with the extra 
+processing power of {ls}. 
+
+This new functionality is made possible by the <<plugins-filters-elastic_integration,elastic_integration filter>> plugin.
+When you include the `elastic_integration` filter in your configuration, {ls} reads certain field values generated by the {agent}, 
+and uses them to apply the transformations from Elastic integrations to further process events before sending them to their 
+configured destinations.
+
+This tutorial walks you through adding the {integrations-docs}/crowdstrike-intro[Crowdstrike integration], using {ls} to 
+remove the `_version` field, and then sending the data to {ess} or self-managed {es}. 
+
+
+[[ea-integrations-prereqs]]
+==== Prerequisites and requirements 
+
+You'll need:
+
+* A working {es} cluster
+* {fleet} server
+* An {agent} configured to send output to {ls}
+* An active Elastic Enterprise https://www.elastic.co/subscriptions[subscription]
+* A user configured with the <<plugins-filters-elastic_integration-minimum_required_privileges,minimum required privileges>>
+
+NOTE: Even though the focus of this tutorial is {Fleet}-managed agents, you can use the `elastic_integration` filter and this 
+general approach with {fleet-guide}/elastic-agent-configuration.html[self-managed agents]. 
+
+
+[[ea-integrations-process-overview]]
+==== Process overview
+
+* <<ea-integrations-fleet>>
+* <<ea-integrations-create-policy>>
+* <<ea-integrations-pipeline>>
+
+[discrete]
+[[ea-integrations-fleet]]
+=== Configure {fleet} to send data from {agent} to {ls}
+
+. For {fleet}-managed agents, go to {kib} and navigate to *Fleet > Settings*.
+
+. Create a new output and specify {ls} as the output type.
+
+. Add the {ls} hosts (domain or IP address/s) that the {agent} should send data to.
+
+. Add the client SSL certificate and the Client SSL certificate key to the configuration.
+
+. Click *Save and apply settings* in the bottom right-hand corner of the page.
+
+[discrete]
+[[ea-integrations-create-policy]]
+=== Create an {agent} policy with the necessary integrations
+
+. In {kib} navigate to *Fleet > Agent* policies, and select *Create agent policy*.
+
+. Give this policy a name, and then select *Advanced options*.
+
+. Change the *Output for integrations* setting to the {ls} output you created.
+
+. Click *Create agent policy*. 
+
+. Select the policy name, and click *Add integration*. 
++ 
+This step takes you to the integrations browser, where you can select an integration that has everything 
+necessary to _integrate_ the data source with your other data in the {stack}.
+We'll use Crowdstrike as our example in this tutorial.
+
+. On the *Crowdstrike* integration overview page, click *Add Crowdstrike* to configure the integration.
+
+. Configure the integration to collect the data you need.
+On step 2 at the bottom of the page (*Where to add this integration?*), make sure that the “Existing hosts” option
+is selected and the Agent policy selected is the Logstash policy we created for our Logstash output. 
+This policy should be selected by default.
+
+. Click *Save and continue*.
++
+You'll have an option to add the {agent} to your hosts. 
+Install the {agent} on the host where you want to collect data (if you have not already done so). 
+Documentation for this process can be
+found here: https://www.elastic.co/guide/en/fleet/current/elastic-agent-installation.html
+
+
+[discrete]
+[[ea-integrations-pipeline]]
+=== Configure {ls} to use the `elastic_integration` filter plugin
+
+. Create a new {logstash-ref}/configuration.html[{ls} pipeline].
+. Be sure to include these plugins:
+
+* `elastic_agent` input
+* `elastic_integration` filter
+* `elasticsearch` output
+
+Note that every event sent from the {agent} to {ls} contains specific meta-fields. 
+{ls} expects events to contain `data_stream.type`, `data_stream.dataset`, and `data_stream.namespace`.
+
+{ls} uses this information and its connection to {es} to determine which integrations to apply to the event before sending the event to its destination output.
+{ls} frequently synchronizes with {es} to ensure that it has the most recent versions of the enabled integrations.
+
+
+[discrete]
+[[ea-integrations-ess-sample]]
+==== Sample configuration: output to {ess}
+
+This sample illustrates using the `elastic_agent` input and the `elastic_integration` filter for processing in {ls}, and then sending the output to {ess}.
+
+Check out the <<plugins-filters-elastic_integration,elastic_integration filter plugin docs>> for the full list of configuration options.
+
+[source,txt]
+-----
+input {
+ elastic_agent { port => 5055 }
+}
+
+filter {
+ elastic_integration {
+ cloud_id => "your-cloud:id"
+ api_key => "api-key"
+ remove_field => ["_version"]
+ }
+}
+
+output {
+ stdout {}
+ elasticsearch {
+ cloud_auth => "elastic:<pwd>"
+ cloud_id => "your-cloud-id"
+ }
+ }
+-----
+
+All processing occurs in {ls} before events are forwarded to {ess}.
+
+[discrete]
+[[ea-integrations-es-sample]]
+==== Sample configuration: output to self-managed {es}
+
+This sample illustrates using the `elastic_agent` input and the `elastic_integration` filter for processing in {ls}, and then sending the output to {es}.
+
+Check out the <<plugins-filters-elastic_integration,elastic_integration filter plugin docs>> for the full list of configuration options.
+
+Check out <<plugins-filters-elastic_integration-minimum_required_privileges>> for more info. 
+
+[source,txt]
+-----
+input {
+ elastic_agent { port => 5055 }
+}
+
+filter {
+ elastic_integration {
+ hosts => "{es-host}:9200"
+ ssl_enabled => true
+ ssl_certificate_authorities => ["/usr/share/logstash/config/certs/ca-cert.pem"]
+ auth_basic_username => "elastic" <1>
+ auth_basic_password => "changeme" <2>
+ remove_field => ["_version"]
+ }
+}
+
+output {
+ stdout {
+ codec => rubydebug # to debug datastream inputs
+ }
+## add elasticsearch
+ elasticsearch {
+ hosts => "{es-host}:9200"
+ password => "changeme"
+ user => "elastic"
+ cacert => "/usr/share/logstash/config/certs/ca-cert.pem"
+ }
+}
+-----
+
+<1> The user credentials that you specify in the `elastic_integration` filter must have sufficient privileges to get information about {es} and the integrations that you are using.
+<2> The user credentials that you specify in the `elastic_integration` filter must have sufficient privileges to get information about {es} and the integrations that you are using.
+
+All processing occurs in {ls} before events are forwarded to {es}.

docs/static/ea-integrations.asciidoc

@@ -8,14 +8,14 @@ You can take advantage of the extensive, built-in capabilities of Elastic {integ
 [[integrations-value]]
 === Elastic {integrations}: ingesting to visualizing 
 
-https://docs.elastic.co/integrations[Elastic {integrations}] provide quick, end-to-end solutions for:
+{integrations-docs}[Elastic {integrations}] provide quick, end-to-end solutions for:
 
 * ingesting data from a variety of data sources, 
 * ensuring compliance with the {ecs-ref}/index.html[Elastic Common Schema (ECS)],
 * getting the data into the {stack}, and 
 * visualizing it with purpose-built dashboards.
 
-{integrations} are available for https://docs.elastic.co/integrations/all_integrations[popular services and platforms], such as Nginx, AWS, and MongoDB, as well as many generic input types like log files.
+{integrations} are available for {integrations-docs}/all_integrations[popular services and platforms], such as Nginx, AWS, and MongoDB, as well as many generic input types like log files.
 Each integration includes pre-packaged assets to help reduce the time between ingest and insights. 
 
 To see available integrations, go to the {kib} home page, and click **Add {integrations}**. 
@@ -78,5 +78,8 @@ output { <3>
 -----
 
 <1> Use `filter-elastic_integration` as the first filter in your pipeline 
-<2> You can use additional filters as long as they follow `filter-elastic_integration`
+<2> You can use additional filters as long as they follow `filter-elastic_integration`. They will have access to the event as-transformed by your enabled integrations.
 <3> Sample config to output data to multiple destinations
+
+
+include::ea-integration-tutorial.asciidoc[]