SDC connector

Introduction

The goal of this connector is to replicate the Secure Data Connector feature offered by Google App Engine on other cloud platforms such as Amazon EC2 and Microsoft Azure. It can also be used during GAE application development to test the usage of the SDC agent as GAE development server doesn't support SDC itself.

The main idea is to reuse the SDC agent provided by Google as an open source project on the server side, installed behind a company firewall to establish a secure tunnel with the Restlet/SDC connector. This connector will then be usable as a regular HTTP client connector.

This project is co-developed by RunMyProcess and Noelios Technologies.

Features

High priority

  • Allow pre-emptive authentication of authorization of SDC agent connections
  • Verify that broken tunnels are properly reused

Medium priority

  • Increase the scalability of the connector using non blocking NIO
  • Load balance for the same account among several SDC agents

Low priority

  • Support resource filtering rules
  • Support OAuth Signed Fetch
  • Support older agent versions

Analysis

Notes

  • When issuing client-side HTTP requests in GAE/J, the "use_intranet" header must be added with an "yes" value
  • The SDC agent supports two ways to proxy
    • HTTP(S) proxying 
    • SOCKS (v4 and v5) proxying
      • lower level proxy protocol supporting both TCP and UDP
      • relies on RFC 1929 for authentication

Protocol buffer schema

The SDC client (inside intranet) exchanges encrypted messages with the SDC server (in the cloud)

  • FrameInfo
    • int64 sequence
    • Type type (SOCKET_DATA, REGISTRATION, HEALTH_CHECK, AUTHORIZATION, FETCH_REQUEST, SOCKET_SESSION)
    • bytes payload
    • string sessionId
  • SocketDataInfo
    • int64 connectionId
    • State state (START, CONTINUE, CLOSE)
    • bytes segment
  • AuthorizationInfo
    • For requests
      • string email
      • AuthType authType (PASSWORD)
      • string password
    • For responses
      • ResultCode result (OK, ACCESS_DENIED, ACCESS_DENIED_CAPTCHA_REQUIRED_TO_UNLOCK, SERVER_ERROR)
      • string statusMessage
  • ResourceKey
    • string ip
    • int32 port
    • int64 key
  • RegistrationInfo
    • string xml
    • string statusMessage
    • ResultCode result (OK, FAILED)
    • ServerSuppliedConf serverSuppliedConf
  • ServerSuppliedConf
    • healthCheckTimeout
    • healthCheckWakeUpInterval
    • For encryption / decryption of fetch / socket requests
      • string sessionId
      • string keyAlgo
      • bytes keyBytes
  • HealthCheckInfo
    • int64 timeStamp
    • server provided configuration parameters
      • Source source (CLIENT, SERVER)
      • Type type (REQUEST, RESPONSE)
  • MessageHeader
    • string key
    • string value
  • FetchRequest
    • string id : the unique call ID
    • string resource : the target HTTP URI
    • string strategy : the HTTP client to use ("URLConnection", "HttpClient", etc.)
    • MessageHeader headers : the HTTP request headers
    • bytes contents : the entity content
  • FetchReply
    • string id : the unique call ID
    • int32 status : the HTTP status code
    • MessageHeader headers : the HTTP response headers
    • bytes contents : the entity content
    • int64 latency : ?
  • SocketSessionRequest
    • SocketSessionVerb verb (CREATE, CONNECT, CLOSE)
    • bytes socketHandle
    • string hostname
    • int32 port
    • MessageHeader headers
    • int64 timeout
  • SocketSessionReply
    • SocketSessionVerb verb
    • bytes socketHandle
    • Status status
    • string hostname
    • int32 port
    • MessageHeader headers
    • int64 latency
  • SocketSessionData
    • bytes socketHandle
    • bytes data
    • int64 streamOffset
    • bool close
  • RegistrationRequestV4
    • string agentId
    • int32 socksServerPort
    • int32 healthCheckPort
    • string healthCheckGadgetUser
    • ResourceKey resourceKey
    • string resourcesXml
  • RegistrationResponseV4
    • string statusMessage
    • ResultCode result (OK, ERRORS_IN_REQUEST, SERVER_ERROR)
    • ServerSuppliedConf serverSuppliedConf

References

Design

  • Usable like a regular HTTP client connector
  • Require the use of the "SDC" protocol name to differentiate requests that must go through the SDC tunnel from regular HTTP requests
  • Request object must set the "protocol" property to "SDC" in order for them to be dispatched properly as the target URI is not discriminating enough)
  • Initially, any SDC agent can connect, how to match those connections, the client requests must have matching identifier and secret (see right after)
  • Initially a challenge response with "SDC" scheme with the following values:
    • identifier = {sdcLogin}@{sdcDomain}
    • secret = {sdcPassword}
    • this allows a proper matching of SDC connections established by SDC agents and application user requests

Implementation

Comments (0)