Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement protocol plugin and rpcv2 protocol #264

Merged
merged 75 commits into from
Feb 22, 2025
Merged
Show file tree
Hide file tree
Changes from 16 commits
Commits
Show all changes
75 commits
Select commit Hold shift + click to select a range
a57acd6
Add protocol base class and draft rpcv2 protocol implementation
jterapin Feb 3, 2025
4d3a7f4
Add example implementation of codegenerated plugin
jterapin Feb 3, 2025
036d2d8
Update smithy model to use rpcv2 trait for testing
jterapin Feb 3, 2025
fb17908
Add projections
jterapin Feb 3, 2025
c7c672c
Merge decaf into branch
jterapin Feb 4, 2025
787f763
Update RPCV2 protocol
jterapin Feb 5, 2025
37c2508
Add protocols weld
jterapin Feb 6, 2025
578478e
Update protocol weld
jterapin Feb 6, 2025
c168be0
Update protocol weld to include a TODO
jterapin Feb 6, 2025
2d098e4
Add more TODO guidance
jterapin Feb 6, 2025
885ae8b
Update protocol plugin
jterapin Feb 6, 2025
20abd06
Update protocol plugin and other components
jterapin Feb 6, 2025
aa367c8
Update handler syntax
jterapin Feb 6, 2025
589feaa
Appease specs
jterapin Feb 6, 2025
8e4c0f3
Update projections
jterapin Feb 6, 2025
03e7699
Remove base class
jterapin Feb 6, 2025
318b0cd
Remove unnecessary method
jterapin Feb 6, 2025
c29eb20
Merge branch 'decaf' into add_cbor_protocol
jterapin Feb 7, 2025
d5413e6
Update protocol modules
jterapin Feb 7, 2025
68713d1
Update documentation
jterapin Feb 7, 2025
5418ba0
Update endpoint specs to handle protocol plugin
jterapin Feb 7, 2025
1f475c2
Move handler classes to smithy-client
jterapin Feb 10, 2025
42114fa
Updates based on feedbacks
jterapin Feb 10, 2025
2656ba7
Sync fixtures
jterapin Feb 10, 2025
86c777d
Update handlers
jterapin Feb 10, 2025
2014c1c
Add rbs file
jterapin Feb 10, 2025
1a26319
Revert "Sync fixtures"
jterapin Feb 10, 2025
1ed8b04
Add protocol plugin specs
jterapin Feb 10, 2025
f7922a9
Update fixture model to recongize config file
jterapin Feb 10, 2025
a4ef87c
update projections
jterapin Feb 10, 2025
f4cb90d
Revert "Update fixture model to recongize config file"
jterapin Feb 10, 2025
fd361e9
Fix config flag for fixtures
jterapin Feb 10, 2025
659a1ea
Fix documentation
jterapin Feb 11, 2025
f3360c3
Merge decaf into branch
jterapin Feb 11, 2025
7952f0e
Merge branch 'decaf' into add_cbor_protocol
jterapin Feb 11, 2025
b4354a9
Update plugin docs
jterapin Feb 11, 2025
44f1898
Fix config documentation
jterapin Feb 11, 2025
4b1ed95
Fix specs based on merged changes
jterapin Feb 11, 2025
15cf0aa
Handle endpoints
jterapin Feb 11, 2025
9a9afb6
Resolve failures relating to content-type and accept headers
jterapin Feb 11, 2025
3f35c47
Add shapes change coverage
jterapin Feb 11, 2025
69208d9
Merge branch 'decaf' into add_cbor_protocol
jterapin Feb 11, 2025
0edce81
Fix codec deserialization
jterapin Feb 11, 2025
046227d
Add possible rbs fix
jterapin Feb 11, 2025
0687fd4
Add header fixes
jterapin Feb 11, 2025
7a9a8f3
Update rbs
jterapin Feb 11, 2025
e05f941
Merge branch 'decaf' into add_cbor_protocol
jterapin Feb 14, 2025
3bc1683
Merge branch 'decaf' into add_cbor_protocol
jterapin Feb 17, 2025
8e201a6
Merge decaf into branch
jterapin Feb 17, 2025
0ae5b87
Adding member names to shape
jterapin Feb 17, 2025
36fe4a5
Update based on refactor
jterapin Feb 17, 2025
c175478
Merge decaf into branch
jterapin Feb 17, 2025
936d4ec
Temp fix
jterapin Feb 17, 2025
d3875d1
Add to reference service shape name
jterapin Feb 18, 2025
eb3255a
Fix failures related to protocol tests
jterapin Feb 18, 2025
c5adb00
Fix top model changes
jterapin Feb 18, 2025
a65325d
Fix rbs
jterapin Feb 18, 2025
cebe5c4
Appease rubocop
jterapin Feb 18, 2025
70dd666
Update syntax
jterapin Feb 20, 2025
52a8d15
Update helpful commands
jterapin Feb 20, 2025
fcb34b1
Update schema and its shapes
jterapin Feb 20, 2025
bd0d912
Update cbor
jterapin Feb 20, 2025
f7358e5
Merge branch 'decaf' into add_cbor_protocol
jterapin Feb 21, 2025
402f07a
Clean up README
jterapin Feb 21, 2025
86995e0
Update Protocol Plugin to hold generic handlers
jterapin Feb 21, 2025
8a29c33
Update based on feedbacks
jterapin Feb 21, 2025
668d9b0
Add DummyWeldProtocol
jterapin Feb 21, 2025
cafe0a9
Fix failure
jterapin Feb 21, 2025
37ca72d
Update existing spec const
jterapin Feb 21, 2025
7d590d8
Fix protocol test failures
jterapin Feb 21, 2025
3ec2dd0
Add members by name to shapes with members
jterapin Feb 22, 2025
75837df
Remove dup method
jterapin Feb 22, 2025
7ba096f
Refactor protocol weld tests
mullermp Feb 22, 2025
8ea5ea0
Refactor weld testing with protocol welds
mullermp Feb 22, 2025
0ab8632
Fix build
mullermp Feb 22, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions gems/smithy-client/lib/smithy-client.rb
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,10 @@
require_relative 'smithy-client/cbor'
require_relative 'smithy-client/codecs/cbor'

# protocols
require_relative 'smithy-client/protocols'
require_relative 'smithy-client/protocols/rpc_v2'

module Smithy
# Base module for a generated Smithy gem.
module Client
Expand Down
10 changes: 10 additions & 0 deletions gems/smithy-client/lib/smithy-client/protocols.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# frozen_string_literal: true

module Smithy
module Client
# TODO: Documentation
# All the generic protocols
module Protocols
end
end
end
57 changes: 57 additions & 0 deletions gems/smithy-client/lib/smithy-client/protocols/rpc_v2.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
# frozen_string_literal: true

module Smithy
module Client
module Protocols
# RPCV2 protocol
# TODO:
# * Refactor methods to handle eventstreams
# * Decide whether we should recognize query traits here
# * Add Documentation
class RPCv2
SHAPE_ID = 'smithy.protocols#rpcv2Cbor'

def initialize(options = {})
@query_compatible = options[:query_compatible]
end

def build_request(context)
codec = Client::Codecs::CBOR.new(setting(context))
context.request.body = codec.serialize(context.params, context.operation.input)
context.request.http_method = 'POST'
apply_headers(context)
end

def parse_response(context)
output_shape = context.operation.output
codec = Client::Codecs::CBOR.new(setting(context))
codec.deserialize(context.response.body, output_shape, output_shape.type)
end

# TODO: Implement once errors are supported
def parse_error(_context, _response); end

private

def apply_headers(context)
context.request.headers['X-Amzn-Query-Mode'] = 'true' if query_compatible?(context)
context.request.headers['Smithy-Protocol'] = 'rpc-v2-cbor'
context.request.headers['Content-Type'] = 'application/cbor'
# TODO: Implement Content-Length Plugin/Handler
context.request.headers['Content-Length'] = context.request.body.size
end

def setting(context)
{}.tap do |h|
h[:query_compatible] = true if query_compatible?(context)
end
end

def query_compatible?(context)
@query_compatible ||
context.config.schema.service.traits.one? { |k, _v| k == 'aws.protocols#awsQuery' }
end
end
end
end
end
6 changes: 6 additions & 0 deletions gems/smithy/lib/smithy/generators/client.rb
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,12 @@ def code_generated_plugins
require_relative: true,
source: Views::Client::EndpointPlugin.new(@plan).render
)
e.yield "lib/#{@gem_name}/plugins/protocol.rb", Views::Client::Plugin.new(
class_name: "#{@plan.module_name}::Plugins::Protocol",
require_path: 'plugins/protocol',
require_relative: true,
source: Views::Client::ProtocolPlugin.new(@plan).render
)
end
end

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,8 @@ module <%= module_name %>
<%= p.param %>: <%= p.value %>,
<% end -%>
)
# TODO: Revise once spec_helper is finalized
client.config.protocol = double(build_request: nil, parse_response: nil)
<% if test_case.expect_error? -%>
expect do
client.<%= operation_input.operation_name %>(
Expand Down
73 changes: 73 additions & 0 deletions gems/smithy/lib/smithy/templates/client/protocol_plugin.erb
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
# frozen_string_literal: true

# This is generated code!

module <%= module_name %>
module Plugins
# Protocol plugin - allows user to configure protocol on client.
# @api private
class Protocol < Smithy::Client::Plugin
option(
:protocol,
<% if protocol -%>
doc_default: '<%= protocol %>',
doc_type: '<%= protocol %>',
docstring: <<~DOCS) do |_cfg|
Allows you to overwrite default protocol. The given protocol
must provide the following functionalities:
- `build_request`
- `parse_response`
- `parse_error`
See existing protocols within Smithy::Client::Protocols for examples.
DOCS
<%= protocol %>.new
end
<% else -%>
default: nil,
docstring: <<~DOCS)
This configuration is required to build requests and parse responses.
In Smithy, a protocol is a named set of rules that defines the syntax
and semantics of how a client and server communicate. The given protocol
must provide the following functionalities:
- `build_request`
- `parse_response`
- `parse_error`
See existing protocols within Smithy::Client::Protocols for examples.
DOCS
<% end -%>

# @api private
class Build < Smithy::Client::Handler
def call(context)
context.config.protocol.build_request(context)
@handler.call(context)
end
end

# @api private
class Parse < Smithy::Client::Handler
def call(context)
resp = @handler.call(context)
context.config.protocol.parse_response(context)
resp
end
end

# @api private
class Error < Smithy::Client::Handler
def call(context)
@handler.call(context).on(300..599) do |response|
context.config.protocol.parse_error(context, response)
end
end
end

def add_handlers(handlers, config)
handlers.add(Build)
handlers.add(Parse)
# TODO: Requires error handling to be implemented
# handlers.add(Error, step: :sign)
end
end
end
end
1 change: 1 addition & 0 deletions gems/smithy/lib/smithy/views/client.rb
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ module Client; end
require_relative 'client/gemspec'
require_relative 'client/module'
require_relative 'client/module_rbs'
require_relative 'client/protocol_plugin'
require_relative 'client/rubocop_yml'
require_relative 'client/shapes'
require_relative 'client/shapes_rbs'
Expand Down
4 changes: 4 additions & 0 deletions gems/smithy/lib/smithy/views/client/client.rb
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,10 @@ def module_name
@plan.module_name
end

def service_id
@plan.service.keys.first
end

def gem_name
@plan.gem_name
end
Expand Down
44 changes: 44 additions & 0 deletions gems/smithy/lib/smithy/views/client/protocol_plugin.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# frozen_string_literal: true

module Smithy
module Views
module Client
# @api private
class ProtocolPlugin < View
def initialize(plan)
@plan = plan
@model = plan.model
super()
end

attr_reader :plan

def module_name
@plan.module_name
end

def protocol
return if default_protocol.nil?

weld_protocols[default_protocol]
end

private

def service_traits
@service_traits ||= @plan.service.values.first['traits']
end

def weld_protocols
@weld_protocols ||= @plan.welds.map(&:protocols).reduce({}, :merge)
end

def default_protocol
return if weld_protocols.empty? && service_traits.nil?

weld_protocols.keys.find { |k| service_traits.key?(k) }
end
end
end
end
end
8 changes: 8 additions & 0 deletions gems/smithy/lib/smithy/weld.rb
Original file line number Diff line number Diff line change
Expand Up @@ -62,5 +62,13 @@ def endpoint_function_bindings
def plugins
{}
end

# Called to construct the protocol plugin. Any protocols defined here will be
# merged with other protocols. The key is the protocol trait ID and the value
# is the fully qualified class name of the protocol.
# @return [Hash<String, String>] protocols for use in protocol plugin.
def protocols
{}
end
end
end
1 change: 1 addition & 0 deletions gems/smithy/lib/smithy/welds.rb
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

require_relative 'welds/endpoints'
require_relative 'welds/plugins'
require_relative 'welds/protocols'
require_relative 'welds/rubocop'

module Smithy
Expand Down
27 changes: 27 additions & 0 deletions gems/smithy/lib/smithy/welds/protocols.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# frozen_string_literal: true

require 'smithy-client/protocols/rpc_v2'

module Smithy
module Welds
# Provides map of protocol trait id and its Ruby class name.
# TODO: Update Welds to have a functionality to control ordering since there
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reason why weld ordering is important.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can add another method (later) for protocol ordering, and in aws-sdk-core, we have a plugin that defines the order (it would also have a reference to any in smithy-client).

# is a priority ordered list of protocols and a requirement that SDK MUST
# select the first entry in their priority ordered list that is also supported
# by the service. Generic code generation MUST accept configuration of this priority
# priority ordered list for use.
# The priority order is as follows (within AWS context):
# * Smithy RPCv2 CBOR
# * AWS JSON 1.0
# * AWS JSON 1.1
# * REST JSON
# * REST XML
# * AWS/Query
# * EC2/Query
class Protocols < Weld
def protocols
{ 'smithy.protocols#rpcv2Cbor' => Smithy::Client::Protocols::RPCv2 }
end
end
end
end
3 changes: 3 additions & 0 deletions model/weather.smithy
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,10 @@ $version: "2"

namespace example.weather

use smithy.protocols#rpcv2Cbor

/// Provides weather forecasts.
@rpcv2Cbor
@paginated(inputToken: "nextToken", outputToken: "nextToken", pageSize: "pageSize")
service Weather {
version: "2006-03-01"
Expand Down
9 changes: 9 additions & 0 deletions projections/weather/lib/weather/client.rb
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
# This is generated code!

require_relative 'plugins/endpoint'
require_relative 'plugins/protocol'
require 'smithy-client/plugins/logging'
require 'smithy-client/plugins/raise_response_errors'
require 'smithy-client/plugins/response_target'
Expand All @@ -15,6 +16,7 @@ class Client < Smithy::Client::Base
self.schema = Shapes::SCHEMA

add_plugin(Weather::Plugins::Endpoint)
add_plugin(Weather::Plugins::Protocol)
add_plugin(Smithy::Client::Plugins::Logging)
add_plugin(Smithy::Client::Plugins::RaiseResponseErrors)
add_plugin(Smithy::Client::Plugins::ResponseTarget)
Expand Down Expand Up @@ -86,6 +88,13 @@ class Client < Smithy::Client::Base
# @option options [Logger] :logger
# The Logger instance to send log messages to. If this option is not set,
# logging is disabled.
# @option options [Smithy::Client::Protocols::RPCv2] :protocol (Smithy::Client::Protocols::RPCv2)
# Allows you to overwrite default protocol. The given protocol
# must provide the following functionalities:
# - `build_request`
# - `parse_response`
# - `parse_error`
# See existing protocols within Smithy::Client::Protocols for examples.
# @option options [Boolean] :raise_response_errors (true)
# When `true`, response errors are raised. When `false`, the error is placed on the
# output in the {Smithy::Client::Output#error error accessor}.
Expand Down
59 changes: 59 additions & 0 deletions projections/weather/lib/weather/plugins/protocol.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
# frozen_string_literal: true

# This is generated code!

module Weather
module Plugins
# Protocol plugin - allows user to configure protocol on client.
# @api private
class Protocol < Smithy::Client::Plugin
option(
:protocol,
doc_default: 'Smithy::Client::Protocols::RPCv2',
doc_type: 'Smithy::Client::Protocols::RPCv2',
docstring: <<~DOCS) do |_cfg|
Allows you to overwrite default protocol. The given protocol
must provide the following functionalities:
- `build_request`
- `parse_response`
- `parse_error`
See existing protocols within Smithy::Client::Protocols for examples.
DOCS
Smithy::Client::Protocols::RPCv2.new
end

# @api private
class Build < Smithy::Client::Handler
def call(context)
context.config.protocol.build_request(context)
@handler.call(context)
end
end

# @api private
class Parse < Smithy::Client::Handler
def call(context)
resp = @handler.call(context)
context.config.protocol.parse_response(context)
resp
end
end

# @api private
class Error < Smithy::Client::Handler
def call(context)
@handler.call(context).on(300..599) do |response|
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately s3 has errors at 200 level, so this may need to be 0..599 or some way to control/customize this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah that makes sense - we should do some brainstorm work when we start looking into error handling.

context.config.protocol.parse_error(context, response)
end
end
end

def add_handlers(handlers, _config)
handlers.add(Build)
handlers.add(Parse)
# TODO: Requires error handling to be implemented
# handlers.add(Error, step: :sign)
end
end
end
end
2 changes: 1 addition & 1 deletion projections/weather/lib/weather/shapes.rb
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ module Shapes
schema.service = ServiceShape.new(
id: 'example.weather#Weather',
version: '2006-03-01',
traits: { 'smithy.api#paginated' => { 'inputToken' => 'nextToken', 'outputToken' => 'nextToken', 'pageSize' => 'pageSize' } }
traits: { 'smithy.api#paginated' => { 'inputToken' => 'nextToken', 'outputToken' => 'nextToken', 'pageSize' => 'pageSize' }, 'smithy.protocols#rpcv2Cbor' => {} }
)
schema.add_operation(:get_city, OperationShape.new do |operation|
operation.id = 'example.weather#GetCity'
Expand Down
Loading
Loading