anyone else integrated NewRelic sidecar?

[erpadmin]

I'm working on loading up the newrelic logging/firelens/fluentbit image as a sidecar. via copilot. I need help on translating the firelensConfiguration for the sidecar and the logConfiguration section to manifest format. If anyone has a example manifest that may help in this regard, please share.

Example fargate task definition, per NewRelic documentation

{
  "family": "newrelic-firelens",
  "networkMode": "awsvpc",
  "requiresCompatibilities": ["FARGATE"],
  "containerDefinitions": [
    // FireLens log router container
    {
      "essential": true,
      // Image below is New Relic's Fluent Bit output plugin available on ECR
      "image": "533243300146.dkr.ecr.us-east-2.amazonaws.com/newrelic/logging-firelens-fluentbit",
      "name": "log_router",
      "firelensConfiguration": {
        "type": "fluentbit",
        "options": {
          "enable-ecs-log-metadata": "true"
        }
      }
    },
    // Application container
    {
      "essential": true,
      "name": "webserver",
      // Application image goes here
      "image": "nginx",
      "cpu": 512,
      "memoryReservation": 1024,
      "portMappings": [
        {
          "containerPort": 5000
        }
      ],
      "environment": [
        {
          "name": "VERSION",
          "value": "V1"
        }
      ],
      // New Relic Fluent Bit output configuration
      "logConfiguration": {
        "logDriver": "awsfirelens",
        "options": {
          "Name": "newrelic"
        },
        "secretOptions": [
          {
            "name": "apiKey",
            "valueFrom": "arn:aws:secretsmanager:region:aws_account_id:secret:secret_name-AbCdEf"
          }
        ]
      }
    }
  ],
  // Use your own role here
  "executionRoleArn": "arn:aws:iam::XXXXXXXXXXXX:role/ecsTaskExecutionRole",
  "taskRoleArn": "arn:aws:iam::XXXXXXXXXXXX:role/ecsTaskExecutionRole",
  "cpu": "1 vcpu",
  "memory": "2 gb"
}

References:
https://docs.newrelic.com/docs/logs/forward-logs/aws-firelens-plugin-log-forwarding/
https://aws.github.io/copilot-cli/docs/developing/sidecars/
#875

[erpadmin]

Looks like I may need to forego the sidecar block and use this logging block

# In the manifest.
logging:
  # The Fluent Bit image. (Optional, we'll use "public.ecr.aws/aws-observability/aws-for-fluent-bit:stable" by default)
  image: <image URL>
  # The configuration options to send to the FireLens log driver. (Optional)
  destination:
    <config key>: <config value>
  # Whether to include ECS metadata in logs. (Optional, default to true)
  enableMetadata: <true|false>
  # Secret to pass to the log configuration. (Optional)
  secretOptions:
    <key>: <value>
  # The full config file path in your custom Fluent Bit image. (Optional)
  configFilePath: <config file path>
  # Environment variables for the sidecar container. (Optional)
  variables:
    <key>: <value>
  # Secrets to expose to the sidecar container. (Optional)
  secrets:
    <key>: <value>

[erpadmin]

Working config (switched to use nginx in the Dockerfile since busybox wasn't producing output)

name: busybox
type: Backend Service

cpu: 256 
memory: 512 
count: 1
exec: true

environments:
  qa: 
    image:
      build: Dockerfile
    variables:
      APP: 'nginx'
    logging:
      image: 533243300146.dkr.ecr.us-east-1.amazonaws.com/newrelic/logging-firelens-fluentbit
      destination:
        Name: newrelic
      enableMetadata: false
      secretOptions:
         apiKey: /copilot/applications/busybox/environments/qa/newrelic-infra/ecs/license-key

NRQL: FROM Log select * WHERE plugin.type='fluent-bit'

[alonssro]

Hey! @erpadmin @iamhopaul123 I'm just trying to integrate new relic with copilot

name: dashboard-backend
type: Load Balanced Web Service

# Distribute traffic to your service.
http:
  # Requests to this path will be forwarded to your service.
  # To match all requests you can use the "/" path.
  path: 'app'
  # You can specify a custom health check path. The default is "/".
  healthcheck: '/app/health'

# Configuration for your containers and service.
image:
  # Docker build arguments. For additional overrides: https://aws.github.io/copilot-cli/docs/manifest/lb-web-service/#image-build
  build: Dockerfile
  # Port exposed through your container to route traffic to it.
  port: 3000

cpu: 256 # Number of CPU units for the task.
memory: 512 # Amount of memory in MiB used by the task.
platform: linux/x86_64 # See https://aws.github.io/copilot-cli/docs/manifest/lb-web-service/#platform
count: 1 # Number of tasks that should be running in your service.
exec: true # Enable running commands in your container.
network:
  connect: true # Enable Service Connect for intra-environment traffic between services.

environments:
  dev:
    cpu: 512
    memory: 1024
    count:
      range: 0-2
      response_time: 4s
  qa:
    cpu: 512
    memory: 1024
    count:
      range: 0-2
      response_time: 4s
  prod:
    cpu: 1024
    memory: 2048
    count:
      range: 1-4
      cpu_percentage: 85
      requests: 250
      response_time: 3s
    logging:
      image: 533243300146.dkr.ecr.us-east-1.amazonaws.com/newrelic/logging-firelens-fluentbit
      destination:
        Name: newrelic
      enableMetadata: true
      secretOptions:
        apiKey: /copilot/${COPILOT_APPLICATION_NAME}/${COPILOT_ENVIRONMENT_NAME}/secrets/new-relic

It's only showing logs and nothing else. I'm doing something wrong?

[iamhopaul123]

Hello @erpadmin. I'm glad you've found your way out! Please let us know if there's anything we could help with.