How to generate SDKs for a REST API powered by Amazon API Gateway

Part of our attachmentAV offering is an API, powered by Amazon API Gateway (REST APIs), allowing developers to integrate virus and malware scanning into their applications. To increase discoverability and simplify integration we decided to build software develpment kits (SDKs) for popular programming languages.

Learn from my journey of finding the best way to generate SDKs for JavaScript, Java, Python, and TypeScript for a REST API.

Built-in function: Generate SDKs for REST APIs

First, I tried the built-in functionality Generate SDKs for REST APIs in API Gateway.

Generating the SDK was simple and fast. However, the result was mediocre. From a developer point of view, using the SDK did not provide significant convience compared to sending plain HTTP request. I was missing an layer of abstraction to simplifiy API usage.

Smithy: Interface Definition Language (IDL)

Second, I looked into Smithy, an Interface Defintion Language developed by AWS to generate their SDKs. Who, if not AWS, must know how to generate SDKs for APIs.

The concept of an extensible, typesafe, protocol agnostic IDL in combination with tools to build client SDKs for various programming languages resonated with me. I found the layer of abstraction, that I was missing while generating SDKs with Amazon API Gateway.

The following snippet shows the model, that I used to build clients for the attachmentAV API.

$version: "2.0"

namespace com.attachmentav.developer

use aws.protocols#restJson1
use aws.api#service

@title("attachmentAV - Virus and Malware Scan API (SaaS)")
@restJson1
@httpApiKeyAuth(name: "x-api-key", in: "header")
@service(sdkId: "VirusMalwareScanSaaS")
service VirusMalwareScanSaaS {
  version: "2006-03-01"
  operations: [CreateSyncDownloadScan, CreateSyncBinaryScan]
}

structure ScanResult {
    
  @required
  @notProperty
  status: String,

  @notProperty
  finding: String,

  @required
  @notProperty
  size: Long,

  @notProperty
  realfiletype: String

}

map DownloadHeaders {
  key: String,
  value: String
}

structure CreateSyncDownloadScanInput {
  @required
  download_url: String,
  download_headers: DownloadHeaders
}

@idempotent
@http(method: "POST", uri: "/v1/scan/sync/download")
operation CreateSyncDownloadScan {
  input: CreateSyncDownloadScanInput,
  output: ScanResult
}

structure CreateSyncBinaryScanInput {
  @httpPayload
  content: Blob
}

@idempotent
@http(method: "POST", uri: "/v1/scan/sync/binary")
operation CreateSyncBinaryScan {
  input: CreateSyncBinaryScanInput,
  output: ScanResult
}

However, I ran into two problems with Smithy. On the one hand, Smithy is clearly focused on generating SDKs for AWS services and therefore makes assumptions about endpoint URLs, authentication mechanisms, and more by default. Using Smithy outside these defaults was tricky for me. On the other hand, most client generators are not production-ready yet and are marked as developer preview. That led to generated code that was not compiling or did not work as intended.

OpenAPI Generator: Generate from OpenAPI documents

Third, after getting advice from Sebastian, Anton, Luciano on LinkedIn I looked into the popular OpenAPI Generator project.

As we use an OpenAPI document to deploy the API Gateway on AWS, giving OpenAPI Generator a try was pretty simple. All I needed to do was to hand over the OpenAPI document to the generator.

But, the result was not very promising. The usability of the generated SDK was pretty bad.

The following idea brought the turnaround: define a seperate OpenAPI document with the aim of generating an SDK that is easy to use. I removed some resources and methods, modified the resource definitions, and more.

The following snippet shows the OpenAPI document optimized to generate SDKs.

openapi: 3.0.0
info:
  description: 'Scan files for viruses, trojans, and other kinds of malware.'
  title: attachmentAV
  version: 1.0.0
servers:
- url: https://eu.developer.attachmentav.com/v1
security:
- apiKeyAuth: []
- bearerAuth: []
tags:
- name: attachmentAV
paths:
  /scan/sync/binary:
    post:
      description: 'Upload a file, scan the file, and return the scan result.'
      tags: [attachmentAV]
      requestBody:
        content:
          application/octet-stream:
            schema:
              format: binary
              type: string
        required: true
      responses:
        '200':
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ScanResult'
          description: Success
  /scan/sync/download:
    post:
      description: 'Download a file from a remote location (HTTP/HTTPS), scan the file, and return the scan result.'
      tags: [ attachmentAV ]
      requestBody:
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/SyncDownloadScanRequest'
        required: true
      responses:
        '200':
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ScanResult'
          description: Success
  /scan/sync/s3:
    post:
      description: 'Download a file from S3, scan the file, and return the scan result. A bucket policy is required to grant attachmentAV access to the S3 objects.'
      tags: [attachmentAV]
      requestBody:
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/SyncS3ScanRequest'
        required: true
      responses:
        '200':
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ScanResult'
          description: Success
components:
  schemas:
    ScanResult:
      example:
        size: size
        realfiletype: realfiletype
        finding: finding
        status: status
      properties:
        status:
          type: string
        finding:
          type: string
        size:
          type: string
        realfiletype:
          type: string
      type: object
    SyncDownloadScanRequest:
      properties:
        download_url:
          type: string
        download_headers:
          additionalProperties:
            type: string
          type: object
      required:
      - download_url
      type: object
    SyncS3ScanRequest:
      properties:
        bucket:
          type: string
        key:
          type: string
        version:
          type: string
      required:
      - bucket
      - key
      type: object
  securitySchemes:
    apiKeyAuth:
      in: header
      name: x-api-key
      type: apiKey
    bearerAuth:
      type: http
      scheme: bearer

I’m pleased with the results (see attachmentav-sdk-js, attachmentav-sdk-java, attachmentav-sdk-python, and attachmentav-sdk-ts).

The following snippet shows how to scan a file for viruses and malware.

import { AttachmentAVApi, Configuration } from '@widdix/attachmentav-sdk-ts';
import * as fs from 'fs';

const config = new Configuration({
  apiKey: '<API_KEY_PLACEHOLDER>'
});

const api = new AttachmentAVApi(config);

async function scanFile() {
  const fileBuffer = fs.readFileSync('./demo.txt');
  const blob = new Blob([fileBuffer]);
  const scanResult = await api.scanSyncBinaryPost({
    body: blob
  });
  console.log('Sync binary scan result:', scanResult);
}

scanFile();

Looking for a way to scan files for viruses, trojans and other kinds of malware. Try the Virus and Malware Scan API by attachmentAV!

Summary

Building SDKs for an API increases discoverability and simplifies integration. Implementing and maintaining SDKs by writing code manually is a huge effort. For us, using the OpenAPI Generator with an optimized OpenAPI document brought the best results.