md.stdout.cli 1.0.0

Std.Out

Captures output of a program to assist with debugging.
This project is tailored towards AWS services, and is not suitable for general purpose diagnostics.

Tracing

Pulls data from various sources, and displays them:

• CloudWatch: Gathers related messages across log streams, and groups.
• S3: Download assets files.
• DynamoDB: Load related records.

CLI

CloudWatch

cw --key appname --cid b6408f5a-6893-4fb7-b996-3946371ab57f

--key: The name of the configuration in app settings, that defines the log groups to query, and general filter rules.
--cid: The Correlation Id to filter the logs by.

S3

s3 --key appname --cid b6408f5a-6893-4fb7-b996-3946371ab57f

--key: The name of the configuration in app settings, that defines the bucket, and path prefix.
--cid: The Correlation Id is part of (or all) of the key, the target files are found under the prefix + correlation id.

DynamoDB

db --key appname --pk b6408f5a-6893-4fb7-b996-3946371ab57f --sk 2022-01-01

--key: The name of the configuration in app settings, that defines the table name, and index to use.
--pk: The Partition Key for an item.
--sk: The Sort Key for an item. If not provided, all sks found under the pk are returned.

AppSettings

{
  "CloudWatch": {
    "Defaults": {
      "Limit": "25",
      "RelativeHours": "1",
      "IsPresentFieldName": "isStructuredLog",
      "CorrelationIdFieldName": "eventProperties.correlationId",
      "Fields": [
        "@timestamp",
        "level",
        "message"
      ],
      "Filters": [
        {
          "Field": "level",
          "Value": "INFO"
        },
        {
          "Field": "eventProperties.customerId",
          "Value": "12345678"
        }
      ]
    },
    "Sources": {
      "AppName": {
        "LogGroups": [
          "/aws/lambda/lambda-one",
          "/aws/lambda/lambda-two"
        ]
      },
      "AnotherAppName": {
        "LogGroups": [
          "/aws/lambda/lambda-three"
        ],
        "Filters": [
          {
            "Field": "level",
            "Value": "ERROR"
          }
        ]
      }
    }
  }
}

CloudWatch

Defaults are applied to all Sources that don't override the property value with their own.
In this example AnotherAppName overrides the Filters value from Defaults.
The "app names" under Sources are matched to the --key command line argument.

cw --key appname --cid 3ee9222f-ed70-475f-8fdc-ee56d1f439da

If sensible defaults can be applied to all sources, then you would only need to set the LogGroups for each logical "app".
Otherwise you can have custom settings for each app under Sources.

• LogGroups: An array of log group names from AWS CloudWatch (required).
• Limit: The maximum number of logs to return for a query (optional: 25).
• RelativeHours: The number of hours to look backwards from "now" (optional: 1).
• IsPresentFieldName: Selects logs with a particular field that must exist (optional: omitted from query).
• CorrelationIdFieldName: The field name that contains an Id, that groups all logs together for a particular request (optional: omitted from query).
• Fields: The CloudWatch fields to select from the query (optional: @timestamp, @message).
• Filters: Clauses to add to the query, each filter will be in the form: "and key = value" (optional: omitted from query).

Credits

• Icon made by Vitaly Gorbachev from Flaticon
• Standard Out Visualization generated from chatgpt (DALL.E / OpenAI).

Changelog

1.0.0

• Created console app project, and readme file.