chiu
/
google-api-ruby-client
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge pull request #564 from google/v0.10 

Merge v0.10 branch (now 0.11)
This commit is contained in:
Steve Bazyl 2017-04-03 12:30:21 -07:00 committed by GitHub
parent 233ee09abc e4edc1584e
commit 8ef5c84c78
34 changed files with 661 additions and 597 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

30
.travis.yml
View File

@ -1,27 +1,21 @@
language: ruby
sudo: false
rvm:
- 2.3.0
- 2.2
- 2.1
- 2.3.1
- 2.2.5
- 2.0.0
- jruby-9000
- 2.1
- jruby-9.0.5.0
env:
global:
- JRUBY_OPTS='-X-C -J-Xmx1024m -J-XX:+UseConcMarkSweepGC'
matrix:
- RAILS_VERSION="~>3.2"
- RAILS_VERSION="~>4.0.0"
- RAILS_VERSION="~>4.1.0"
- RAILS_VERSION="~>4.2.0"
- RAILS_VERSION="~>4.2.0"
- RAILS_VERSION="~>5.0.0"
matrix:
exclude:
- rvm: 2.0.0
env: RAILS_VERSION="~>4.2.0"
script: "bundle exec rake spec:all"
before_install:
- sudo apt-get update
- sudo apt-get install idn
- gem update bundler
- env: RAILS_VERSION="~>5.0.0"
rvm: 2.0.0
- env: RAILS_VERSION="~>5.0.0"
rvm: 2.1
before_install: gem install bundler
notifications:
email:
recipients:

20
CHANGELOG.md
View File

@ -1,7 +1,24 @@
# 0.11.0
* *Breaking change* - Fix handling of large numbers during code generation. Previously the
uint64/int64 formats were passed through as strings. They are now coerced to/from Fixnum/Bignum types
* *Breaking change* - No longer normalize unicode strings in URI templates. Mostly affects
Cloud Storage, but other APIs with unicode strings in paths may be affected. Old behavior
can be restored using the `normalize_unicode` request option.
* *Breaking change* -- Moved timeout options from `RequestOptions` to `ClientOptions`
* Remove Hurley as dependency. May cause minor breaking changes if directly accessing the underlying
client connection.
* Drop compatibility with Rails 3.x since that is no longer supported by the Rails community.
* Upgrade mime-types to 3.0
* Move Thor & ActiveSupport to development dependencies. Using the code gengerator
now requires using the Bundle file or install the gem with dev dependencies.
* Treat 429 status codes as rate limit errors
* Fix a potential download corruption if download interrupted and retried against a URL
that does not return partial content.
# 0.10.3
* Regenerate APIs
* Enable additional API:
* `acceleratedmobilepageurl:v1``
* `acceleratedmobilepageurl:v1`
* `appengine:v1`
* `clouderrorreporting:v1beta1`
* `cloudfunctions:v1`
@ -103,7 +120,6 @@
* Reduce memory footprint used by mimetypes library
* Fix bug with pagination when items collection is nil
# 0.9.9
* Add monitoring v3, regenerate APIs
* Add samples for sheets, bigquery

12
Gemfile
View File

@ -6,22 +6,20 @@ gemspec
group :development do
gem 'bundler', '~> 1.7'
gem 'rake', '~> 10.0'
gem 'rake', '~> 11.2'
gem 'rspec', '~> 3.1'
gem 'json_spec', '~> 1.1'
gem 'webmock', '~> 1.21'
gem 'simplecov', '~> 0.9'
gem 'coveralls', '~> 0.7.11'
gem 'rubocop', '~> 0.29'
gem 'webmock', '~> 2.1'
gem 'simplecov', '~> 0.12'
gem 'coveralls', '~> 0.8'
gem 'rubocop', '~> 0.42.0'
gem 'launchy', '~> 2.4'
gem 'dotenv', '~> 2.0'
gem 'fakefs', '~> 0.6', require: "fakefs/safe"
gem 'google-id-token', '~> 1.3'
gem 'os', '~> 0.9'
gem 'rmail', '~> 1.1'
gem 'sinatra', '~> 1.4'
gem 'redis', '~> 3.2'
gem 'activesupport', '>= 3.2', '< 5.0'
end
platforms :jruby do

33
MIGRATING.md
View File

@ -1,3 +1,36 @@
# Migrating from version`0.10` to `0.11`
## Unicode normalization
The client no longer normalizes unicode strings in path parameters. This may affect
some applications using multibyte strings that were previously normalized.:
To restore the previous behavior, set the following option prior to creating a service.
```ruby
ClientOptions.default.normalize_unicode = true
```
## Type change for large numbers
Previously, types declared as 64 bit numbers were mapped to strings. These are now mapped to
`Fixednum`/`Bignum`.
## Timeouts
Timeout options have been moved from `RequestOptions` to `ClientOptions`.
Old | New
----------------------------------|-----------------
`RequestOptions.open_timeout_sec` | `ClentOptions.open_timeout_sec`
`RequestOptions.timeout_sec` | `ClentOptions.read_timeout_sec`
`RequestOptions.timeout_sec` | `ClentOptions.send_timeout_sec`
## Batch requests across services no longer supported
It is no longer possible to combine multiple services (e.g. Gail & Drive)
in a batch request. If batching requests that span services, group
requests for each service in their own batch request.
# Migrating from version `0.9.x` to `0.10`
Only one minor breaking change was introduced in the `to_json` method due to a version bump for the `representable` gem from `2.3` to `3.0`. If you used the `skip_undefined` in `to_json`, you should replace that with `user_options: { skip_undefined: true }`.

10
README.md
View File

@ -191,6 +191,14 @@ file = drive.create_file(file) # Raises ArgumentError: unknown keywords: id, tit
file = drive.create_file(file, {}) # Returns a Drive::File instance
```
### Using raw JSON
To handle JSON serialization or deserialization in the application, set `skip_serialization` or
or `skip_deserializaton` options respectively. When setting `skip_serialization` in a request,
the body object must be a string representing the serialized JSON.
When setting `skip_deserialization` to true, the response from the API will likewise
be a string containing the raw JSON from the server.
## Authorization
@ -283,7 +291,7 @@ The second is to set the environment variable `GOOGLE_API_USE_RAILS_LOGGER` to a
## Samples
See the [samples](samples) for examples on how to use the client library for various
See the [samples](https://github.com/google/google-api-ruby-client/tree/master/samples) for examples on how to use the client library for various
services.
Contributions for additional samples are welcome. See [CONTRIBUTING](CONTRIBUTING.md).

1
Rakefile
View File

@ -1,2 +1,3 @@
require "bundler/gem_tasks"
task default: :spec

8
bin/generate-api
View File

@ -1,6 +1,12 @@
#!/usr/bin/env ruby
require 'thor'
begin
require 'thor'
rescue LoadError => e
puts "Thor is required. Please install the gem with development dependencies."
exit 1
end
require 'open-uri'
require 'google/apis/discovery_v1'
require 'logger'

0
dl.rb Normal file
View File

10
google-api-client.gemspec
View File

@ -22,12 +22,10 @@ Gem::Specification.new do |spec|
spec.add_runtime_dependency 'representable', '~> 3.0'
spec.add_runtime_dependency 'retriable', '>= 2.0', '< 4.0'
spec.add_runtime_dependency 'addressable', '~> 2.3'
spec.add_runtime_dependency 'mime-types', '>= 1.6'
spec.add_runtime_dependency 'hurley', '~> 0.1'
spec.add_runtime_dependency 'addressable', '>= 2.5.1'
spec.add_runtime_dependency 'mime-types', '>= 3.0'
spec.add_runtime_dependency 'googleauth', '~> 0.5'
spec.add_runtime_dependency 'httpclient', '~> 2.7'
spec.add_runtime_dependency 'memoist', '~> 0.11'
spec.add_runtime_dependency 'httpclient', '>= 2.8.1', '< 3.0'
spec.add_development_dependency 'thor', '~> 0.19'
spec.add_development_dependency 'activesupport', '>= 4.2', '< 5.1'
end

11
lib/google/apis/core/api_command.rb
View File

@ -54,8 +54,12 @@ module Google
def prepare!
query[FIELDS_PARAM] = normalize_fields_param(query[FIELDS_PARAM]) if query.key?(FIELDS_PARAM)
if request_representation && request_object
header[:content_type] ||= JSON_CONTENT_TYPE
self.body = request_representation.new(request_object).to_json(user_options: { skip_undefined: true })
header['Content-Type'] ||= JSON_CONTENT_TYPE
if options && options.skip_serialization
self.body = request_object
else
self.body = request_representation.new(request_object).to_json(user_options: { skip_undefined: true })
end
end
super
end
@ -71,6 +75,7 @@ module Google
# noinspection RubyUnusedLocalVariable
def decode_response_body(content_type, body)
return super unless response_representation
return super if options && options.skip_deserialization
return super if content_type.nil?
return nil unless content_type.start_with?(JSON_CONTENT_TYPE)
instance = response_class.new
@ -82,7 +87,7 @@ module Google
#
# @param [Fixnum] status
# HTTP status code of response
# @param [Hurley::Header] header
# @param [Hash] header
# HTTP response headers
# @param [String] body
# HTTP response body

42
lib/google/apis/core/base_service.rb
View File

@ -19,11 +19,9 @@ require 'google/apis/core/api_command'
require 'google/apis/core/batch'
require 'google/apis/core/upload'
require 'google/apis/core/download'
require 'google/apis/core/http_client_adapter'
require 'google/apis/options'
require 'googleauth'
require 'hurley'
require 'hurley/addressable'
require 'httpclient'
module Google
module Apis
@ -86,6 +84,8 @@ module Google
# Base service for all APIs. Not to be used directly.
#
class BaseService
include Logging
# Root URL (host/port) for the API
# @return [Addressable::URI]
attr_accessor :root_url
@ -103,7 +103,7 @@ module Google
attr_accessor :batch_path
# HTTP client
# @return [Hurley::Client]
# @return [HTTPClient]
attr_accessor :client
# General settings
@ -205,7 +205,7 @@ module Google
end
# Get the current HTTP client
# @return [Hurley::Client]
# @return [HTTPClient]
def client
@client ||= new_client
end
@ -375,19 +375,33 @@ module Google
end
# Create a new HTTP client
# @return [Hurley::Client]
# @return [HTTPClient]
def new_client
client = Hurley::Client.new
client.connection = Google::Apis::Core::HttpClientAdapter.new unless client_options.use_net_http
client.request_options.timeout = request_options.timeout_sec
client.request_options.open_timeout = request_options.open_timeout_sec
client.request_options.proxy = client_options.proxy_url
client.request_options.query_class = Hurley::Query::Flat
client.ssl_options.ca_file = File.join(Google::Apis::ROOT, 'lib', 'cacerts.pem')
client.header[:user_agent] = user_agent
client = ::HTTPClient.new
client.transparent_gzip_decompression = true
client.proxy = client_options.proxy_url if client_options.proxy_url
if client_options.open_timeout_sec
client.connect_timeout = client_options.open_timeout_sec
end
if client_options.read_timeout_sec
client.receive_timeout = client_options.read_timeout_sec
end
if client_options.send_timeout_sec
client.send_timeout = client_options.send_timeout_sec
end
client.follow_redirect_count = 5
client.default_header = { 'User-Agent' => user_agent }
client.debug_dev = logger if client_options.log_http_requests
client
end
# Build the user agent header
# @return [String]
def user_agent

31
lib/google/apis/core/batch.rb
View File

@ -25,19 +25,19 @@
# See the License for the specific language governing permissions and
# limitations under the License.
require 'hurley'
require 'google/apis/core/multipart'
require 'google/apis/core/http_command'
require 'google/apis/core/upload'
require 'google/apis/core/download'
require 'google/apis/core/composite_io'
require 'addressable/uri'
require 'securerandom'
module Google
module Apis
module Core
# Wrapper request for batching multiple calls in a single server request
class BatchCommand < HttpCommand
BATCH_BOUNDARY = 'RubyApiBatchRequest'.freeze
MULTIPART_MIXED = 'multipart/mixed'
# @param [symbol] method
@ -81,7 +81,7 @@ module Google
parts.each_index do |index|
response = deserializer.to_http_response(parts[index])
outer_header = response.shift
call_id = header_to_id(outer_header[:content_id]) || index
call_id = header_to_id(outer_header['Content-ID'].first) || index
call, callback = @calls[call_id]
begin
result = call.process_response(*response) unless call.nil?
@ -106,17 +106,16 @@ module Google
fail BatchError, 'Cannot make an empty batch request' if @calls.empty?
serializer = CallSerializer.new
multipart = Multipart.new(boundary: BATCH_BOUNDARY, content_type: MULTIPART_MIXED)
multipart = Multipart.new(content_type: MULTIPART_MIXED)
@calls.each_index do |index|
call, _ = @calls[index]
content_id = id_to_header(index)
io = serializer.to_upload_io(call)
multipart.add_upload(io, content_id: content_id)
io = serializer.to_part(call)
multipart.add_upload(io, content_type: 'application/http', content_id: content_id)
end
self.body = multipart.assemble
header[:content_type] = multipart.content_type
header[:content_length] = "#{body.length}"
header['Content-Type'] = multipart.content_type
super
end
@ -155,24 +154,20 @@ module Google
# Serializes a command for embedding in a multipart batch request
# @private
class CallSerializer
HTTP_CONTENT_TYPE = 'application/http'
##
# Serialize a single batched call for assembling the multipart message
#
# @param [Google::Apis::Core::HttpCommand] call
# the call to serialize.
# @return [Hurley::UploadIO]
# @return [IO]
# the serialized request
def to_upload_io(call)
def to_part(call)
call.prepare!
parts = []
parts << build_head(call)
parts << build_body(call) unless call.body.nil?
length = parts.inject(0) { |a, e| a + e.length }
Hurley::UploadIO.new(Hurley::CompositeReadIO.new(length, *parts),
HTTP_CONTENT_TYPE,
'ruby-api-request')
Google::Apis::Core::CompositeIO.new(*parts)
end
protected
@ -201,7 +196,7 @@ module Google
#
# @param [String] call_response
# the response to parse.
# @return [Array<(Fixnum, Hurley::Header, String)>]
# @return [Array<(Fixnum, Hash, String)>]
# Status, header, and response body.
def to_http_response(call_response)
outer_header, outer_body = split_header_and_body(call_response)
@ -218,10 +213,10 @@ module Google
#
# @param [String] response
# the response to parse.
# @return [Array<(Hurley::Header, String)>]
# @return [Array<(HTTP::Message::Headers, String)>]
# the header and the body, separately.
def split_header_and_body(response)
header = Hurley::Header.new
header = HTTP::Message::Headers.new
payload = response.lstrip
while payload
line, payload = payload.split(/\n/, 2)

97
lib/google/apis/core/composite_io.rb Normal file
View File

@ -0,0 +1,97 @@
# Copyright 2015 Google Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Copyright 2015 Google Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
require 'google/apis/core/http_command'
require 'google/apis/core/upload'
require 'google/apis/core/download'
require 'addressable/uri'
require 'securerandom'
module Google
module Apis
module Core
class CompositeIO
def initialize(*ios)
@ios = ios.flatten
@pos = 0
@index = 0
@sizes = @ios.map(&:size)
end
def read(length = nil, buf = nil)
buf = buf ? buf.replace('') : ''
begin
io = @ios[@index]
break if io.nil?
result = io.read(length)
if result
buf << result
if length
length -= result.length
break if length == 0
end
end
@index += 1
end while @index < @ios.length
buf.length > 0 ? buf : nil
end
def size
@sizes.reduce(:+)
end
alias_method :length, :size
def pos
@pos
end
def pos=(pos)
fail ArgumentError, "Position can not be negative" if pos < 0
@pos = pos
new_index = nil
@ios.each_with_index do |io,idx|
size = io.size
if pos <= size
new_index ||= idx
io.pos = pos
pos = 0
else
io.pos = size
pos -= size
end
end
@index = new_index unless new_index.nil?
end
def rewind
self.pos = 0
end
end
end
end
end

44
lib/google/apis/core/download.rb
View File

@ -12,7 +12,6 @@
# See the License for the specific language governing permissions and
# limitations under the License.
require 'google/apis/core/multipart'
require 'google/apis/core/api_command'
require 'google/apis/errors'
require 'addressable/uri'
@ -22,7 +21,7 @@ module Google
module Core
# Streaming/resumable media download support
class DownloadCommand < ApiCommand
RANGE_HEADER = 'range'
RANGE_HEADER = 'Range'
OK_STATUS = [200, 201, 206]
# File or IO to write content to
@ -58,7 +57,7 @@ module Google
# of file content.
#
# @private
# @param [Hurley::Client] client
# @param [HTTPClient] client
# HTTP client
# @yield [result, err] Result or error if block supplied
# @return [Object]
@ -66,40 +65,45 @@ module Google
# @raise [Google::Apis::ClientError] The request is invalid and should not be retried without modification
# @raise [Google::Apis::AuthorizationError] Authorization is required
def execute_once(client, &block)
response = client.get(@download_url || url) do |req|
apply_request_options(req)
check_if_rewind_needed = false
if @offset > 0
logger.debug { sprintf('Resuming download from offset %d', @offset) }
req.header[RANGE_HEADER] = sprintf('bytes=%d-', @offset)
check_if_rewind_needed = true
end
req.on_body(*OK_STATUS) do |res, chunk|
check_status(res.status_code, chunk) unless res.status_code.nil?
if check_if_rewind_needed && res.status_code != 206
request_header = header.dup
apply_request_options(request_header)
check_if_rewind_needed = false
if @offset > 0
logger.debug { sprintf('Resuming download from offset %d', @offset) }
request_header[RANGE_HEADER] = sprintf('bytes=%d-', @offset)
check_if_rewind_needed = true
end
http_res = client.get(url.to_s,
query: query,
header: request_header,
follow_redirect: true) do |res, chunk|
status = res.http_header.status_code.to_i
if OK_STATUS.include?(status)
if check_if_rewind_needed && status != 206
# Oh no! Requested a chunk, but received the entire content
# Attempt to rewind the stream
@download_io.rewind
check_if_rewind_needed = false
end
logger.debug { sprintf('Writing chunk (%d bytes)', chunk.length) }
@offset += chunk.length
# logger.debug { sprintf('Writing chunk (%d bytes, %d total)', chunk.length, bytes_read) }
@download_io.write(chunk)
@download_io.flush
@offset += chunk.length
end
end
# Since the on_body block only runs on success, check status again just in case it failed
check_status(response.status_code, response.body) unless OK_STATUS.include?(response.status_code.to_i)
@download_io.flush
if @close_io_on_finish
result = nil
else
result = @download_io
end
check_status(http_res.status.to_i, http_res.header, http_res.body)
success(result, &block)
rescue => e
@download_io.flush
error(e, rethrow: true, &block)
end
end

82
lib/google/apis/core/http_client_adapter.rb
View File

@ -1,82 +0,0 @@
require 'httpclient'
require 'hurley'
require 'hurley/client'
module Google
module Apis
module Core
# HTTPClient adapter for Hurley.
class HttpClientAdapter
def call(request)
client = ::HTTPClient.new
configure_client(client, request)
begin
::Hurley::Response.new(request) do |res|
http_res = client.request(request.verb.to_s.upcase, request.url.to_s, nil, request.body_io, request.header.to_hash, false) do |http_res, chunk|
copy_response(http_res, res)
res.receive_body(chunk)
end
copy_response(http_res, res)
end
rescue ::HTTPClient::TimeoutError, Errno::ETIMEDOUT
raise ::Hurley::Timeout, $!
rescue ::HTTPClient::BadResponseError => err
if err.message.include?('status 407')
raise ::Hurley::ConnectionFailed, %{407 "Proxy Authentication Required "}
else
raise Hurley::ClientError, $!
end
rescue Errno::ECONNREFUSED, EOFError
raise ::Hurley::ConnectionFailed, $!
rescue => err
if defined?(OpenSSL) && OpenSSL::SSL::SSLError === err
raise Hurley::SSLError, err
else
raise
end
end
end
def copy_response(http_res, res)
unless res.status_code
res.status_code = http_res.status.to_i
http_res.header.all.each do |(k,v)|
res.header[k] = v
end
end
end
def configure_client(client, request)
client.transparent_gzip_decompression = true
if request.options.proxy
proxy = request.options.proxy
client.proxy = sprintf('%s:%d', proxy.host, proxy.port)
if proxy.user && proxy.password
client.set_proxy_auth proxy.user, proxy.password
end
end
if request.options.timeout
client.connect_timeout = request.options.timeout
client.receive_timeout = request.options.timeout
client.send_timeout = request.options.timeout
end
if request.options.open_timeout
client.connect_timeout = request.options.open_timeout
client.send_timeout = request.options.open_timeout
end
ssl_config = client.ssl_config
ssl_opts = request.ssl_options
ssl_config.verify_mode = ssl_opts.openssl_verify_mode
ssl_config.cert_store = ssl_opts.openssl_cert_store
ssl_config.add_trust_ca ssl_opts.ca_file if ssl_opts.ca_file
ssl_config.add_trust_ca ssl_opts.ca_path if ssl_opts.ca_path
ssl_config.client_cert = ssl_opts.openssl_client_cert if ssl_opts.openssl_client_cert
ssl_config.client_key = ssl_opts.openssl_client_key if ssl_opts.openssl_client_key
ssl_config.verify_depth = ssl_opts.verify_depth if ssl_opts.verify_depth
end
end
end
end
end

87
lib/google/apis/core/http_command.rb
View File

@ -17,9 +17,6 @@ require 'addressable/template'
require 'google/apis/options'
require 'google/apis/errors'
require 'retriable'
require 'hurley'
require 'hurley/addressable'
require 'hurley_patches'
require 'google/apis/core/logging'
require 'pp'
@ -41,7 +38,7 @@ module Google
attr_accessor :url
# HTTP headers
# @return [Hurley::Header]
# @return [Hash]
attr_accessor :header
# Request body
@ -53,7 +50,7 @@ module Google
attr_accessor :method
# HTTP Client
# @return [Hurley::Client]
# @return [HTTPClient]
attr_accessor :connection
# Query params
@ -75,7 +72,7 @@ module Google
self.url = url
self.url = Addressable::Template.new(url) if url.is_a?(String)
self.method = method
self.header = Hurley::Header.new
self.header = Hash.new
self.body = body
self.query = {}
self.params = {}
@ -83,7 +80,7 @@ module Google
# Execute the command, retrying as necessary
#
# @param [Hurley::Client] client
# @param [HTTPClient] client
# HTTP client
# @yield [result, err] Result or error if block supplied
# @return [Object]
@ -142,8 +139,12 @@ module Google
# @private
# @return [void]
def prepare!
header.update(options.header) if options && options.header
self.url = url.expand(params) if url.is_a?(Addressable::Template)
normalize_unicode = true
if options
header.update(options.header) if options.header
normalize_unicode = options.normalize_unicode
end
self.url = url.expand(params, nil, normalize_unicode) if url.is_a?(Addressable::Template)
url.query_values = query.merge(url.query_values || {})
if allow_form_encoding?
@ -154,6 +155,9 @@ module Google
else
@form_encoded = false
end
self.body = '' unless self.body
end
# Release any resources used by this command
@ -166,7 +170,7 @@ module Google
#
# @param [Fixnum] status
# HTTP status code of response
# @param [Hurley::Header] header
# @param [Hash] header
# Response headers
# @param [String, #read] body
# Response body
@ -177,7 +181,7 @@ module Google
# @raise [Google::Apis::AuthorizationError] Authorization is required
def process_response(status, header, body)
check_status(status, header, body)
decode_response_body(header[:content_type], body)
decode_response_body(header['Content-Type'].first, body)
end
# Check the response and raise error if needed
@ -185,7 +189,7 @@ module Google
# @param [Fixnum] status
# HTTP status code of response
# @param
# @param [Hurley::Header] header
# @param [Hash] header
# HTTP response headers
# @param [String] body
# HTTP response body
@ -201,11 +205,14 @@ module Google
when 200...300
nil
when 301, 302, 303, 307
message ||= sprintf('Redirect to %s', header[:location])
message ||= sprintf('Redirect to %s', header['Location'])
raise Google::Apis::RedirectError.new(message, status_code: status, header: header, body: body)
when 401
message ||= 'Unauthorized'
raise Google::Apis::AuthorizationError.new(message, status_code: status, header: header, body: body)
when 429
message ||= 'Rate limit exceeded'
raise Google::Apis::RateLimitError.new(message, status_code: status, header: header, body: body)
when 304, 400, 402...500
message ||= 'Invalid request'
raise Google::Apis::ClientError.new(message, status_code: status, header: header, body: body)
@ -251,7 +258,16 @@ module Google
# @raise [StandardError] if no block
def error(err, rethrow: false, &block)
logger.debug { sprintf('Error - %s', PP.pp(err, '')) }
err = Google::Apis::TransmissionError.new(err) if err.is_a?(Hurley::ClientError) || err.is_a?(SocketError)
if err.is_a?(HTTPClient::BadResponseError)
begin
res = err.res
check_status(res.status.to_i, res.header, res.body)
rescue Google::Apis::Error => e
err = e
end
elsif err.is_a?(HTTPClient::TimeoutError) || err.is_a?(SocketError)
err = Google::Apis::TransmissionError.new(err)
end
block.call(nil, err) if block_given?
fail err if rethrow || block.nil?
end
@ -259,7 +275,7 @@ module Google
# Execute the command once.
#
# @private
# @param [Hurley::Client] client
# @param [HTTPClient] client
# HTTP client
# @return [Object]
# @raise [Google::Apis::ServerError] An error occurred on the server and the request can be retried
@ -269,21 +285,18 @@ module Google
body.rewind if body.respond_to?(:rewind)
begin
logger.debug { sprintf('Sending HTTP %s %s', method, url) }
response = client.send(method, url, body) do |req|
# Temporary workaround for Hurley bug where the connection preference
# is ignored and it uses nested anyway
unless form_encoded?
req.url.query_class = Hurley::Query::Flat
query.each do | k, v|
req.url.query[k] = normalize_query_value(v)
end
end
# End workaround
apply_request_options(req)
end
logger.debug { response.status_code }
logger.debug { response.inspect }
response = process_response(response.status_code, response.header, response.body)
request_header = header.dup
apply_request_options(request_header)
http_res = client.request(method.to_s.upcase,
url.to_s,
query: nil,
body: body,
header: request_header,
follow_redirect: true)
logger.debug { http_res.status }
logger.debug { http_res.inspect }
response = process_response(http_res.status.to_i, http_res.header, http_res.body)
success(response)
rescue => e
logger.debug { sprintf('Caught error %s', e) }
@ -292,18 +305,16 @@ module Google
end
# Update the request with any specified options.
# @param [Hurley::Request] req
# HTTP request
# @param [Hash] header
# HTTP headers
# @return [void]
def apply_request_options(req)
def apply_request_options(req_header)
if options.authorization.respond_to?(:apply!)
options.authorization.apply!(req.header)
options.authorization.apply!(req_header)
elsif options.authorization.is_a?(String)
req.header[:authorization] = sprintf('Bearer %s', options.authorization)
req_header['Authorization'] = sprintf('Bearer %s', options.authorization)
end
req.header.update(header)
req.options.timeout = options.timeout_sec
req.options.open_timeout = options.open_timeout_sec
req_header.update(header)
end
def allow_form_encoding?

6
lib/google/apis/core/json_representation.rb
View File

@ -55,7 +55,7 @@ module Google
def if_fn(name)
ivar_name = "@#{name}".to_sym
lambda do |opts|
if opts[:options][:user_options] and opts[:options][:user_options][:skip_undefined]
if opts[:user_options] && opts[:user_options][:skip_undefined]
if respond_to?(:key?)
self.key?(name) || instance_variable_defined?(ivar_name)
else
@ -72,6 +72,10 @@ module Google
options[:render_filter] = ->(value, _doc, *_args) { value.nil? ? nil : Base64.urlsafe_encode64(value) }
options[:parse_filter] = ->(fragment, _doc, *_args) { Base64.urlsafe_decode64(fragment) }
end
if options[:numeric_string]
options[:render_filter] = ->(value, _doc, *_args) { value.nil? ? nil : value.to_s}
options[:parse_filter] = ->(fragment, _doc, *_args) { fragment.to_i }
end
if options[:type] == DateTime
options[:render_filter] = ->(value, _doc, *_args) { value.nil? ? nil : value.is_a?(DateTime) ? value.rfc3339(3) : value.to_s }
options[:parse_filter] = ->(fragment, _doc, *_args) { DateTime.parse(fragment) }

136
lib/google/apis/core/multipart.rb
View File

@ -12,7 +12,6 @@
# See the License for the specific language governing permissions and
# limitations under the License.
require 'hurley'
module Google
module Apis
@ -21,108 +20,60 @@ module Google
#
# @private
class JsonPart
include Hurley::Multipart::Part
# @return [Fixnum]
# Length of part
attr_reader :length
# @param [String] boundary
# Multipart boundary
# @param [String] value
# JSON content
def initialize(boundary, value, header = {})
@part = build_part(boundary, value)
@length = @part.bytesize
@io = StringIO.new(@part)
# @param [Hash] header
# Additional headers
def initialize(value, header = {})
@value = value
@header = header
end
private
# Format the part
#
# @param [String] boundary
# Multipart boundary
# @param [String] value
# JSON content
# @return [String]
def build_part(boundary, value)
def to_io(boundary)
part = ''
part << "--#{boundary}\r\n"
part << "Content-Type: application/json\r\n"
@header.each do |(k, v)|
part << "#{k}: #{v}\r\n"
end
part << "\r\n"
part << "#{value}\r\n"
part << "#{@value}\r\n"
StringIO.new(part)
end
end
# Part of a multipart request for holding arbitrary content. Modified
# from Hurley::Multipart::FilePart to remove Content-Disposition
# Part of a multipart request for holding arbitrary content.
#
# @private
class FilePart
include Hurley::Multipart::Part
# @return [Fixnum]
# Length of part
attr_reader :length
# @param [String] boundary
# Multipart boundary
# @param [Google::Apis::Core::UploadIO] io
# @param [IO] io
# IO stream
# @param [Hash] header
# Additional headers
def initialize(boundary, io, header = {})
file_length = io.respond_to?(:length) ? io.length : File.size(io.local_path)
@head = build_head(boundary, io.content_type, file_length,
io.respond_to?(:opts) ? io.opts.merge(header) : header)
@length = @head.bytesize + file_length + FOOT.length
@io = Hurley::CompositeReadIO.new(@length, StringIO.new(@head), io, StringIO.new(FOOT))
def initialize(io, header = {})
@io = io
@header = header
@length = io.respond_to?(:size) ? io.size : nil
end
private
# Construct the header for the part
#
# @param [String] boundary
# Multipart boundary
# @param [String] type
# Content type for the part
# @param [Fixnum] content_len
# Length of the part
# @param [Hash] header
# Headers for the part
def build_head(boundary, type, content_len, header)
content_id = ''
if header[:content_id]
content_id = sprintf(CID_FORMAT, header[:content_id])
def to_io(boundary)
head = ''
head << "--#{boundary}\r\n"
@header.each do |(k, v)|
head << "#{k}: #{v}\r\n"
end
sprintf(HEAD_FORMAT,
boundary,
content_len.to_i,
content_id,
header[:content_type] || type,
header[:content_transfer_encoding] || DEFAULT_TR_ENCODING)
head << "Content-Length: #{@length}\r\n" unless @length.nil?
head << "Content-Transfer-Encoding: binary\r\n"
head << "\r\n"
Google::Apis::Core::CompositeIO.new(StringIO.new(head), @io, StringIO.new("\r\n"))
end
DEFAULT_TR_ENCODING = 'binary'.freeze
FOOT = "\r\n".freeze
CID_FORMAT = "Content-ID: %s\r\n"
HEAD_FORMAT = <<-END
--%s\r
Content-Length: %d\r
%sContent-Type: %s\r
Content-Transfer-Encoding: %s\r
\r
END
end
# Helper for building multipart requests
class Multipart
MULTIPART_RELATED = 'multipart/related'
DEFAULT_BOUNDARY = 'RubyApiClientMultiPart'
# @return [String]
# Content type header
@ -135,8 +86,8 @@ Content-Transfer-Encoding: %s\r
def initialize(content_type: MULTIPART_RELATED, boundary: nil)
@parts = []
@boundary = boundary || DEFAULT_BOUNDARY
@content_type = "#{content_type}; boundary=#{boundary}"
@boundary = boundary || Digest::SHA1.hexdigest(SecureRandom.random_bytes(8))
@content_type = "#{content_type}; boundary=#{@boundary}"
end
# Append JSON data part
@ -147,23 +98,26 @@ Content-Transfer-Encoding: %s\r
# Optional unique ID of this part
# @return [self]
def add_json(body, content_id: nil)
header = { :content_id => content_id }
@parts << Google::Apis::Core::JsonPart.new(@boundary, body, header)
header = {}
header['Content-ID'] = content_id unless content_id.nil?
@parts << Google::Apis::Core::JsonPart.new(body, header).to_io(@boundary)
self
end
# Append arbitrary data as a part
#
# @param [Google::Apis::Core::UploadIO] upload_io
# @param [IO] upload_io
# IO stream
# @param [String] content_id
# Optional unique ID of this part
# @return [self]
def add_upload(upload_io, content_id: nil)
header = { :content_id => content_id }
@parts << Google::Apis::Core::FilePart.new(@boundary,
upload_io,
header)
def add_upload(upload_io, content_type: nil, content_id: nil)
header = {
'Content-Type' => content_type || 'application/octet-stream'
}
header['Content-Id'] = content_id unless content_id.nil?
@parts << Google::Apis::Core::FilePart.new(upload_io,
header).to_io(@boundary)
self
end
@ -172,14 +126,8 @@ Content-Transfer-Encoding: %s\r
# @return [IO]
# IO stream
def assemble
@parts << Hurley::Multipart::EpiloguePart.new(@boundary)
ios = []
len = 0
@parts.each do |part|
len += part.length
ios << part.to_io
end
Hurley::CompositeReadIO.new(len, *ios)
@parts << StringIO.new("--#{@boundary}--\r\n\r\n")
Google::Apis::Core::CompositeIO.new(*@parts)
end
end
end

143
lib/google/apis/core/upload.rb
View File

@ -18,58 +18,18 @@ require 'google/apis/core/api_command'
require 'google/apis/errors'
require 'addressable/uri'
require 'tempfile'
begin
require 'mime/types/columnar'
rescue LoadError
# Temporary until next major bump when we can tighten
# dependency to mime-types >-=3.0
require 'mime-types'
end
require 'mime-types'
module Google
module Apis
module Core
# Extension of Hurley's UploadIO to add length accessor
class UploadIO < Hurley::UploadIO
OCTET_STREAM_CONTENT_TYPE = 'application/octet-stream'
# Get the length of the stream
# @return [Fixnum]
def length
io.respond_to?(:length) ? io.length : File.size(local_path)
end
# Create a new instance given a file path
# @param [String, File] file_name
# Path to file
# @param [String] content_type
# Optional content type. If nil, will attempt to auto-detect
# @return [Google::Apis::Core::UploadIO]
def self.from_file(file_name, content_type: nil)
if content_type.nil?
type = MIME::Types.of(file_name)
content_type = type.first.content_type unless type.nil? || type.empty?
end
new(file_name, content_type || OCTET_STREAM_CONTENT_TYPE)
end
# Wraps an IO stream in UploadIO
# @param [#read] io
# IO to wrap
# @param [String] content_type
# Optional content type.
# @return [Google::Apis::Core::UploadIO]
def self.from_io(io, content_type: OCTET_STREAM_CONTENT_TYPE)
new(io, content_type)
end
end
# Base upload command. Not intended to be used directly
# @private
class BaseUploadCommand < ApiCommand
UPLOAD_PROTOCOL_HEADER = 'X-Goog-Upload-Protocol'
UPLOAD_CONTENT_TYPE_HEADER = 'X-Goog-Upload-Header-Content-Type'
UPLOAD_CONTENT_LENGTH = 'X-Goog-Upload-Header-Content-Length'
CONTENT_TYPE_HEADER = 'Content-Type'
# File name or IO containing the content to upload
# @return [String, File, #read]
@ -83,21 +43,29 @@ module Google
# @return [Google::Apis::Core::UploadIO]
attr_accessor :upload_io
# Ensure the content is readable and wrapped in an {{Google::Apis::Core::UploadIO}} instance.
# Ensure the content is readable and wrapped in an IO instance.
#
# @return [void]
# @raise [Google::Apis::ClientError] if upload source is invalid
def prepare!
super
if streamable?(upload_source)
self.upload_io = UploadIO.from_io(upload_source, content_type: upload_content_type)
self.upload_io = upload_source
@close_io_on_finish = false
elsif upload_source.is_a?(String)
self.upload_io = UploadIO.from_file(upload_source, content_type: upload_content_type)
elsif self.upload_source.is_a?(String)
self.upload_io = File.new(upload_source, 'r')
if self.upload_content_type.nil?
type = MIME::Types.of(upload_source)
self.upload_content_type = type.first.content_type unless type.nil? || type.empty?
end
@close_io_on_finish = true
else
fail Google::Apis::ClientError, 'Invalid upload source'
end
if self.upload_content_type.nil? || self.upload_content_type.empty?
self.upload_content_type = 'application/octet-stream'
end
puts self.upload_content_type.inspect
end
# Close IO stream when command done. Only closes the stream if it was opened by the command.
@ -124,13 +92,12 @@ module Google
super
self.body = upload_io
header[UPLOAD_PROTOCOL_HEADER] = RAW_PROTOCOL
header[UPLOAD_CONTENT_TYPE_HEADER] = upload_io.content_type
header[UPLOAD_CONTENT_TYPE_HEADER] = upload_content_type
end
end
# Implementation of the multipart upload protocol
class MultipartUploadCommand < BaseUploadCommand
UPLOAD_BOUNDARY = 'RubyApiClientUpload'
MULTIPART_PROTOCOL = 'multipart'
MULTIPART_RELATED = 'multipart/related'
@ -140,11 +107,11 @@ module Google
# @raise [Google::Apis::ClientError] if upload source is invalid
def prepare!
super
@multipart = Multipart.new(boundary: UPLOAD_BOUNDARY, content_type: MULTIPART_RELATED)
@multipart.add_json(body)
@multipart.add_upload(upload_io)
self.body = @multipart.assemble
header[:content_type] = @multipart.content_type
multipart = Multipart.new
multipart.add_json(body)
multipart.add_upload(upload_io, content_type: upload_content_type)
self.body = multipart.assemble
header['Content-Type'] = multipart.content_type
header[UPLOAD_PROTOCOL_HEADER] = MULTIPART_PROTOCOL
end
end
@ -179,7 +146,7 @@ module Google
#
# @param [Fixnum] status
# HTTP status code of response
# @param [Hurley::Header] header
# @param [HTTP::Message::Headers] header
# Response headers
# @param [String, #read] body
# Response body
@ -189,9 +156,9 @@ module Google
# @raise [Google::Apis::ClientError] The request is invalid and should not be retried without modification
# @raise [Google::Apis::AuthorizationError] Authorization is required
def process_response(status, header, body)
@offset = Integer(header[BYTES_RECEIVED_HEADER]) if header.key?(BYTES_RECEIVED_HEADER)
@upload_url = header[UPLOAD_URL_HEADER] if header.key?(UPLOAD_URL_HEADER)
upload_status = header[UPLOAD_STATUS_HEADER]
@offset = Integer(header[BYTES_RECEIVED_HEADER].first) unless header[BYTES_RECEIVED_HEADER].empty?
@upload_url = header[UPLOAD_URL_HEADER].first unless header[UPLOAD_URL_HEADER].empty?
upload_status = header[UPLOAD_STATUS_HEADER].first
logger.debug { sprintf('Upload status %s', upload_status) }
if upload_status == STATUS_ACTIVE
@state = :active
@ -204,61 +171,69 @@ module Google
super(status, header, body)
end
# Send the start command to initiate the upload
#
# @param [Hurley::Client] client
# HTTP client
# @return [Hurley::Response]
# @raise [Google::Apis::ServerError] Unable to send the request
def send_start_command(client)
logger.debug { sprintf('Sending upload start command to %s', url) }
client.send(method, url, body) do |req|
apply_request_options(req)
req.header[UPLOAD_PROTOCOL_HEADER] = RESUMABLE
req.header[UPLOAD_COMMAND_HEADER] = START_COMMAND
req.header[UPLOAD_CONTENT_LENGTH] = upload_io.length.to_s
req.header[UPLOAD_CONTENT_TYPE_HEADER] = upload_io.content_type
end
request_header = header.dup
apply_request_options(request_header)
request_header[UPLOAD_PROTOCOL_HEADER] = RESUMABLE
request_header[UPLOAD_COMMAND_HEADER] = START_COMMAND
request_header[UPLOAD_CONTENT_LENGTH] = upload_io.size.to_s
request_header[UPLOAD_CONTENT_TYPE_HEADER] = upload_content_type
client.request(method.to_s.upcase,
url.to_s, query: nil,
body: body,
header: request_header,
follow_redirect: true)
rescue => e
raise Google::Apis::ServerError, e.message
end
# Query for the status of an incomplete upload
#
# @param [Hurley::Client] client
# @param [HTTPClient] client
# HTTP client
# @return [Hurley::Response]
# @return [HTTP::Message]
# @raise [Google::Apis::ServerError] Unable to send the request
def send_query_command(client)
logger.debug { sprintf('Sending upload query command to %s', @upload_url) }
client.post(@upload_url, nil) do |req|
apply_request_options(req)
req.header[UPLOAD_COMMAND_HEADER] = QUERY_COMMAND
end
request_header = header.dup
apply_request_options(request_header)
request_header[UPLOAD_COMMAND_HEADER] = QUERY_COMMAND
client.post(@upload_url, body: '', header: request_header, follow_redirect: true)
end
# Send the actual content
#
# @param [Hurley::Client] client
# @param [HTTPClient] client
# HTTP client
# @return [Hurley::Response]
# @return [HTTP::Message]
# @raise [Google::Apis::ServerError] Unable to send the request
def send_upload_command(client)
logger.debug { sprintf('Sending upload command to %s', @upload_url) }
content = upload_io
content.pos = @offset
client.post(@upload_url, content) do |req|
apply_request_options(req)
req.header[UPLOAD_COMMAND_HEADER] = UPLOAD_COMMAND
req.header[UPLOAD_OFFSET_HEADER] = @offset.to_s
end
request_header = header.dup
apply_request_options(request_header)
request_header[UPLOAD_COMMAND_HEADER] = QUERY_COMMAND
request_header[UPLOAD_COMMAND_HEADER] = UPLOAD_COMMAND
request_header[UPLOAD_OFFSET_HEADER] = @offset.to_s
request_header[CONTENT_TYPE_HEADER] = upload_content_type
client.post(@upload_url, body: content, header: request_header, follow_redirect: true)
end
# Execute the upload request once. This will typically perform two HTTP requests -- one to initiate or query
# for the status of the upload, the second to send the (remaining) content.
#
# @private
# @param [Hurley::Client] client
# @param [HTTPClient] client
# HTTP client
# @yield [result, err] Result or error if block supplied
# @return [Object]

2
lib/google/apis/generator/model.rb
View File

@ -51,6 +51,8 @@ module Google
when 'string', 'boolean', 'number', 'integer', 'any'
return 'DateTime' if format == 'date-time'
return 'Date' if format == 'date'
return 'Fixnum' if format == 'int64'
return 'Fixnum' if format == 'uint64'
return TYPE_MAP[type]
when 'array'
return sprintf('Array<%s>', items.generated_type)

2
lib/google/apis/generator/templates/_representation.tmpl
View File

@ -33,7 +33,7 @@ class <%= cls.generated_class_name %>
<% elsif property.type == 'array' -%>
collection :<%= property.generated_name %>, as: '<%= property.name %>'<%= include('representation_type', :lead => ', ', :type => property.items, :api => api) %>
<% else -%>
property :<%= property.generated_name %>,<% if property.format == 'byte' %> :base64 => true,<%end%> as: '<%= property.name %>'<%= include('representation_type', :lead => ', ', :type => property, :api => api) %>
property :<%= property.generated_name %>,<% if ['uint64', 'int64'].include?(property.format) %> :numeric_string => true,<%end%><% if property.format == 'byte' %> :base64 => true,<%end%> as: '<%= property.name %>'<%= include('representation_type', :lead => ', ', :type => property, :api => api) %>
<% end -%>
<% end -%>
<% end -%>

36
lib/google/apis/options.rb
View File

@ -15,18 +15,22 @@
module Google
module Apis
# General options for API requests
ClientOptions = Struct.new(
ClientOptions = Struct.new(
:application_name,
:application_version,
:proxy_url,
:use_net_http)
:open_timeout_sec,
:read_timeout_sec,
:send_timeout_sec,
:log_http_requests)
RequestOptions = Struct.new(
:authorization,
:retries,
:header,
:timeout_sec,
:open_timeout_sec)
:normalize_unicode,
:skip_serialization,
:skip_deserialization)
# General client options
class ClientOptions
@ -36,7 +40,12 @@ module Google
# @return [String] Version of the application, for identification in the User-Agent header
# @!attribute [rw] proxy_url
# @return [String] URL of a proxy server
# @!attribute [rw] log_http_requests
# @return [Boolean] True if raw HTTP requests should be logged
# @!attribute [rw] open_timeout_sec
# @return [Fixnum] How long, in seconds, before failed connections time out
# @!attribute [rw] read_timeout_sec
# @return [Fixnum] How long, in seconds, before requests time out
# Get the default options
# @return [Google::Apis::ClientOptions]
def self.default
@ -50,12 +59,14 @@ module Google
# @return [Signet::OAuth2::Client, #apply(Hash)] OAuth2 credentials
# @!attribute [rw] retries
# @return [Fixnum] Number of times to retry requests on server error
# @!attribute [rw] timeout_sec
# @return [Fixnum] How long, in seconds, before requests time out
# @!attribute [rw] open_timeout_sec
# @return [Fixnum] How long, in seconds, before failed connections time out
# @!attribute [rw] header
# @return [Hash<String,String] Additional HTTP headers to include in requests
# @!attribute [rw] normalize_unicode
# @return [Boolean] True if unicode strings should be normalized in path parameters
# @!attribute [rw] skip_serialization
# @return [Boolean] True if body object should be treated as raw text instead of an object.
# @!attribute [rw] skip_deserialization
# @return [Boolean] True if response should be returned in raw form instead of deserialized.
# Get the default options
# @return [Google::Apis::RequestOptions]
@ -75,11 +86,12 @@ module Google
end
end
ClientOptions.default.use_net_http = false
ClientOptions.default.log_http_requests = false
ClientOptions.default.application_name = 'unknown'
ClientOptions.default.application_version = '0.0.0'
RequestOptions.default.retries = 0
RequestOptions.default.open_timeout_sec = 20
RequestOptions.default.normalize_unicode = false
RequestOptions.default.skip_serialization = false
RequestOptions.default.skip_deserialization = false
end
end

2
lib/google/apis/version.rb
View File

@ -15,7 +15,7 @@
module Google
module Apis
# Client library version
VERSION = '0.10.3'
VERSION = '0.11.0'
# Current operating system
# @private

1
samples/web/Gemfile
View File

@ -2,6 +2,5 @@ source 'https://rubygems.org'
gem 'google-api-client', '~> 0.9'
gem 'google-id-token', '~> 1.3'
gem 'sinatra', '~> 1.4'
gem 'redis', '~> 3.2'
gem 'dotenv'

30
spec/google/apis/core/api_command_spec.rb
View File

@ -15,7 +15,6 @@
require 'spec_helper'
require 'google/apis/core/api_command'
require 'google/apis/core/json_representation'
require 'hurley/test'
RSpec.describe Google::Apis::Core::ApiCommand do
include TestHelpers
@ -64,6 +63,29 @@ RSpec.describe Google::Apis::Core::ApiCommand do
end
end
context('with a raw request body') do
let(:command) do
request = model_class.new
command = Google::Apis::Core::ApiCommand.new(:post, 'https://www.googleapis.com/zoo/animals')
command.request_representation = representer_class
command.request_object = %({"value": "hello"})
command.options.skip_serialization = true
command
end
before(:example) do
stub_request(:post, 'https://www.googleapis.com/zoo/animals')
.to_return(headers: { 'Content-Type' => 'application/json' }, body: %({}))
end
it 'should allow raw JSON if skip_serialization = true' do
command.execute(client)
expect(a_request(:post, 'https://www.googleapis.com/zoo/animals').with do |req|
be_json_eql(%({"value":"hello"})).matches?(req.body)
end).to have_been_made
end
end
context('with a JSON response') do
let(:command) do
command = Google::Apis::Core::ApiCommand.new(:get, 'https://www.googleapis.com/zoo/animals')
@ -86,6 +108,12 @@ RSpec.describe Google::Apis::Core::ApiCommand do
result = command.execute(client)
expect(result.value).to eql 'hello'
end
it 'should return a raw JSON if skip_deserialization true' do
command.options.skip_deserialization = true
result = command.execute(client)
expect(result).to eql %({"value" : "hello"})
end
end
context('with an invalid content-type response') do

26
spec/google/apis/core/batch_spec.rb
View File

@ -15,7 +15,6 @@
require 'spec_helper'
require 'google/apis/core/batch'
require 'google/apis/core/json_representation'
require 'hurley/test'
RSpec.describe Google::Apis::Core::BatchCommand do
include TestHelpers
@ -30,19 +29,20 @@ RSpec.describe Google::Apis::Core::BatchCommand do
let(:post_with_string_command) do
command = Google::Apis::Core::HttpCommand.new(:post, 'https://www.googleapis.com/zoo/animals/2')
command.body = 'Hello world'
command.header[:content_type] = 'text/plain'
command.header['Content-Type'] = 'text/plain'
command
end
let(:post_with_io_command) do
command = Google::Apis::Core::HttpCommand.new(:post, 'https://www.googleapis.com/zoo/animals/3')
command.body = StringIO.new('Goodbye!')
command.header[:content_type] = 'text/plain'
command.header['Content-Type'] = 'text/plain'
command
end
before(:example) do
allow(SecureRandom).to receive(:uuid).and_return('ffe23d1b-e8f7-47f5-8c01-2a30cf8ecb8f')
allow(Digest::SHA1).to receive(:hexdigest).and_return('123abc')
response = <<EOF
--batch123
@ -83,20 +83,20 @@ EOF
command.execute(client)
expected_body = <<EOF.gsub(/\n/, "\r\n")
--RubyApiBatchRequest
Content-Length: 58
Content-ID: <ffe23d1b-e8f7-47f5-8c01-2a30cf8ecb8f+0>
--123abc
Content-Type: application/http
Content-Id: <ffe23d1b-e8f7-47f5-8c01-2a30cf8ecb8f+0>
Content-Length: 58
Content-Transfer-Encoding: binary
GET /zoo/animals/1? HTTP/1.1
Host: www.googleapis.com
--RubyApiBatchRequest
Content-Length: 96
Content-ID: <ffe23d1b-e8f7-47f5-8c01-2a30cf8ecb8f+1>
--123abc
Content-Type: application/http
Content-Id: <ffe23d1b-e8f7-47f5-8c01-2a30cf8ecb8f+1>
Content-Length: 96
Content-Transfer-Encoding: binary
POST /zoo/animals/2? HTTP/1.1
@ -104,10 +104,10 @@ Content-Type: text/plain
Host: www.googleapis.com
Hello world
--RubyApiBatchRequest
Content-Length: 93
Content-ID: <ffe23d1b-e8f7-47f5-8c01-2a30cf8ecb8f+2>
--123abc
Content-Type: application/http
Content-Id: <ffe23d1b-e8f7-47f5-8c01-2a30cf8ecb8f+2>
Content-Length: 93
Content-Transfer-Encoding: binary
POST /zoo/animals/3? HTTP/1.1
@ -115,7 +115,7 @@ Content-Type: text/plain
Host: www.googleapis.com
Goodbye!
--RubyApiBatchRequest--
--123abc--
EOF
expect(a_request(:post, 'https://www.googleapis.com/batch').with(body: expected_body)).to have_been_made

76
spec/google/apis/core/composite_io_spec.rb Normal file
View File

@ -0,0 +1,76 @@
# Copyright 2015 Google Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
require 'spec_helper'
require 'google/apis/core/composite_io'
RSpec.describe Google::Apis::Core::CompositeIO do
shared_examples 'should act like IO' do
it 'should read from all IOs' do
expect(io.read).to eq 'Hello Cruel World'
end
it 'should respond to size' do
expect(io.size).to eq 17
end
it 'should respond to pos=' do
io.pos = 6
expect(io.read).to eq('Cruel World')
end
it 'should reject negative positions' do
expect { io.pos = -1 }.to raise_error(ArgumentError)
end
it 'should return nil if position beyond size' do
io.pos = 20
expect(io.read).to be_nil
end
it 'should be readable after rewinding' do
expect(io.read).to eq 'Hello Cruel World'
expect(io.read).to be_nil
io.rewind
expect(io.read).to eq 'Hello Cruel World'
end
end
context 'with StringIOs' do
let(:io) do
Google::Apis::Core::CompositeIO.new(
StringIO.new("Hello "),
StringIO.new("Cruel "),
StringIO.new("World"))
end
include_examples 'should act like IO'
end
context 'with Files' do
let(:io) do
files = []
dir = Dir.mktmpdir
['Hello ', 'Cruel ', 'World'].each_with_index do |text, index|
name = File.join(dir, "f#{index}")
File.open(name, 'w') { |f| f.write(text) }
files << name
end
Google::Apis::Core::CompositeIO.new(files.map { |name| File.open(name, 'r') })
end
include_examples 'should act like IO'
end
end

1
spec/google/apis/core/download_spec.rb
View File

@ -15,7 +15,6 @@
require 'spec_helper'
require 'google/apis/core/download'
require 'google/apis/core/json_representation'
require 'hurley/test'
require 'tempfile'
require 'tmpdir'

28
spec/google/apis/core/http_command_spec.rb
View File

@ -14,7 +14,6 @@
require 'spec_helper'
require 'google/apis/core/http_command'
require 'hurley/test'
RSpec.describe Google::Apis::Core::HttpCommand do
include TestHelpers
@ -300,4 +299,31 @@ RSpec.describe Google::Apis::Core::HttpCommand do
command.options.retries = 0
expect { command.execute(client) }.to raise_error(Google::Apis::TransmissionError)
end
it 'should raise rate limit error for 429 status codes' do
stub_request(:get, 'https://www.googleapis.com/zoo/animals').to_return(status: [429, ''])
command = Google::Apis::Core::HttpCommand.new(:get, 'https://www.googleapis.com/zoo/animals')
command.options.retries = 0
expect { command.execute(client) }.to raise_error(Google::Apis::RateLimitError)
end
it 'should not normalize unicode values by default' do
stub_request(:get, 'https://www.googleapis.com/Cafe%CC%81').to_return(status: [200, ''])
template = Addressable::Template.new('https://www.googleapis.com/{path}')
command = Google::Apis::Core::HttpCommand.new(:get, template)
command.params[:path] = "Cafe\u0301"
command.options.retries = 0
command.execute(client)
end
it 'should normalize unicode values when requested' do
stub_request(:get, 'https://www.googleapis.com/Caf%C3%A9').to_return(status: [200, ''])
template = Addressable::Template.new('https://www.googleapis.com/{path}')
command = Google::Apis::Core::HttpCommand.new(:get, template)
command.params[:path] = "Cafe\u0301"
command.options.retries = 0
command.options.normalize_unicode = true
command.execute(client)
end
end

14
spec/google/apis/core/json_representation_spec.rb
View File

@ -33,6 +33,7 @@ RSpec.describe Google::Apis::Core::JsonRepresentation do
attr_accessor :date_value
attr_accessor :nil_date_value
attr_accessor :bytes_value
attr_accessor :big_value
attr_accessor :items
attr_accessor :child
attr_accessor :children
@ -51,6 +52,7 @@ RSpec.describe Google::Apis::Core::JsonRepresentation do
property :date_value, as: 'dateValue', type: DateTime
property :nil_date_value, as: 'nullDateValue', type: DateTime
property :bytes_value, as: 'bytesValue', base64: true
property :big_value, as: 'bigValue', numeric_string: true
property :items
property :child, class: klass do
property :value
@ -106,6 +108,10 @@ RSpec.describe Google::Apis::Core::JsonRepresentation do
it 'serializes object collections' do
expect(json).to be_json_eql(%([{"value" : "child"}])).at_path('children')
end
it 'serializes numeric strings' do
expect(json).to be_json_eql(%("1208925819614629174706176")).at_path('bigValue')
end
end
context 'with model object' do
@ -124,6 +130,7 @@ RSpec.describe Google::Apis::Core::JsonRepresentation do
model.child.value = 'child'
model.children = [model.child]
model.nil_date_value = nil
model.big_value = 1208925819614629174706176
model
end
@ -143,6 +150,7 @@ RSpec.describe Google::Apis::Core::JsonRepresentation do
boolean_value_true: true,
boolean_value_false: false,
bytes_value: 'Hello world',
big_value: 1208925819614629174706176,
items: [1, 2, 3],
child: {
value: 'child'
@ -165,6 +173,7 @@ RSpec.describe Google::Apis::Core::JsonRepresentation do
"numericValue": 123,
"dateValue": "2015-05-01T12:00:00+00:00",
"bytesValue": "SGVsbG8gd29ybGQ=",
"bigValue": "1208925819614629174706176",
"items": [1,2,3],
"child": {"value" : "hello"},
"children": [{"value" : "hello"}]
@ -204,5 +213,10 @@ EOF
it 'serializes object collections' do
expect(model.children[0].value).to eql 'hello'
end
it 'deserializes numeric strings' do
expect(model.big_value).to eql 1208925819614629174706176
end
end
end

54
spec/google/apis/core/service_spec.rb
View File

@ -16,8 +16,6 @@ require 'spec_helper'
require 'google/apis/options'
require 'google/apis/core/base_service'
require 'google/apis/core/json_representation'
require 'hurley/test'
require 'ostruct'
RSpec.describe Google::Apis::Core::BaseService do
include TestHelpers
@ -95,6 +93,18 @@ RSpec.describe Google::Apis::Core::BaseService do
end
end
context 'with proxy' do
after(:example) do
Google::Apis::ClientOptions.default.proxy_url = nil
end
it 'should allow proxy URLs as strings' do
Google::Apis::ClientOptions.default.proxy_url = 'http://gateway.example.com:1234'
service.client
end
end
context 'when making simple commands' do
let(:command) { service.send(:make_simple_command, :get, 'zoo/animals', authorization: 'foo') }
@ -190,6 +200,8 @@ EOF
context 'with batch uploads' do
before(:example) do
allow(SecureRandom).to receive(:uuid).and_return('b1981e17-f622-49af-b2eb-203308b1b17d')
allow(Digest::SHA1).to receive(:hexdigest).and_return('outer', 'inner')
response = <<EOF.gsub(/\n/, "\r\n")
--batch123
Content-Type: application/http
@ -227,6 +239,44 @@ EOF
end.to yield_with_args('Hello', nil)
end
it 'should send nested multipart' do
service.batch_upload do |service|
command = service.send(:make_upload_command, :post, 'zoo/animals', {})
command.upload_source = StringIO.new('test')
command.upload_content_type = 'text/plain'
service.send(:execute_or_queue_command, command)
end
expected_body = <<EOF.gsub(/\n/, "\r\n")
--outer
Content-Type: application/http
Content-Id: <b1981e17-f622-49af-b2eb-203308b1b17d+0>
Content-Length: 303
Content-Transfer-Encoding: binary
POST /upload/zoo/animals? HTTP/1.1
Content-Type: multipart/related; boundary=inner
X-Goog-Upload-Protocol: multipart
Host: www.googleapis.com
--inner
Content-Type: application/json
--inner
Content-Type: text/plain
Content-Length: 4
Content-Transfer-Encoding: binary
test
--inner--
--outer--
EOF
expect(a_request(:put, 'https://www.googleapis.com/upload/').with(body: expected_body)).to have_been_made
end
it 'should disallow downloads in batch' do
expect do |b|
service.batch_upload do |service|

73
spec/google/apis/core/upload_spec.rb
View File

@ -15,70 +15,6 @@
require 'spec_helper'
require 'google/apis/core/upload'
require 'google/apis/core/json_representation'
require 'hurley/test'
# TODO: JSON Response decoding
# TODO: Upload from IO
# TODO: Upload from file
RSpec.describe Google::Apis::Core::UploadIO do
context 'from_file' do
let(:upload_io) { Google::Apis::Core::UploadIO.from_file(file) }
context 'with text file' do
let(:file) { File.join(FIXTURES_DIR, 'files', 'test.txt') }
it 'should infer content type from file' do
expect(upload_io.content_type).to eql('text/plain')
end
it 'should allow overriding the mime type' do
io = Google::Apis::Core::UploadIO.from_file(file, content_type: 'application/json')
expect(io.content_type).to eql('application/json')
end
end
context 'with unknown type' do
let(:file) { File.join(FIXTURES_DIR, 'files', 'test.blah') }
it 'should use the default mime type' do
expect(upload_io.content_type).to eql('application/octet-stream')
end
it 'should allow overriding the mime type' do
io = Google::Apis::Core::UploadIO.from_file(file, content_type: 'application/json')
expect(io.content_type).to eql('application/json')
end
it 'should setup length of the stream' do
upload_io = Google::Apis::Core::UploadIO.from_file(file)
expect(upload_io.length).to eq File.size(file)
end
end
end
context 'from_io' do
context 'with i/o stream' do
let(:io) { StringIO.new 'Hello google' }
it 'should setup default content-type' do
upload_io = Google::Apis::Core::UploadIO.from_io(io)
expect(upload_io.content_type).to eql Google::Apis::Core::UploadIO::OCTET_STREAM_CONTENT_TYPE
end
it 'should allow overring the mime type' do
upload_io = Google::Apis::Core::UploadIO.from_io(io, content_type: 'application/x-gzip')
expect(upload_io.content_type).to eq('application/x-gzip')
end
it 'should setup length of the stream' do
upload_io = Google::Apis::Core::UploadIO.from_io(io)
expect(upload_io.length).to eq 'Hello google'.length
end
end
end
end
RSpec.describe Google::Apis::Core::RawUploadCommand do
include TestHelpers
@ -170,21 +106,22 @@ RSpec.describe Google::Apis::Core::MultipartUploadCommand do
before(:example) do
stub_request(:post, 'https://www.googleapis.com/zoo/animals').to_return(body: %(Hello world))
allow(Digest::SHA1).to receive(:hexdigest).and_return('123abc')
end
it 'should send content' do
expected_body = <<EOF.gsub(/\n/, "\r\n")
--RubyApiClientUpload
--123abc
Content-Type: application/json
metadata
--RubyApiClientUpload
Content-Length: 11
--123abc
Content-Type: text/plain
Content-Length: 11
Content-Transfer-Encoding: binary
Hello world
--RubyApiClientUpload--
--123abc--
EOF
command.execute(client)

5
spec/google/apis/errors_spec.rb
View File

@ -80,9 +80,8 @@ RSpec.describe Google::Apis::Error do
context '@cause is falsy' do
before do
subject.class.superclass.any_instance.stub(:backtrace) do
"super class's #backtrace called"
end
expect_any_instance_of(subject.class.superclass).to receive(:backtrace).and_return(
"super class's #backtrace called")
end
it "calls super class's #backtrace" do

103
third_party/hurley_patches.rb vendored
View File

@ -1,103 +0,0 @@
#
# Copyright (c) 2015 Rick Olson
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
#
require 'hurley'
require 'hurley/client'
require 'hurley/connection'
require 'net/https'
# Temporary monkey patch for streaming downloads. These are fixed in HEAD,
# but pending a 0.3 release.
if Hurley::VERSION == '0.2'
module Hurley
class Response
def location
@location ||= begin
return unless loc = @header[:location]
verb = STATUS_FORCE_GET.include?(status_code) ? :get : request.verb
statuses, receiver = request.send(:body_receiver)
new_request = Request.new(verb, request.url.join(Url.parse(loc)), request.header, request.body, request.options, request.ssl_options)
new_request.on_body(*statuses, &receiver) unless receiver.is_a?(Hurley::BodyReceiver)
new_request
end
end
end
class Connection
def call(request)
net_http_connection(request) do |http|
begin
Response.new(request) do |res|
perform_request(http, request, res)
# net/http only raises exception on 407 with ssl...?
if res.status_code == 407
raise ConnectionFailed, %(407 "Proxy Authentication Required")
end
end
rescue *NET_HTTP_EXCEPTIONS => err
if defined?(OpenSSL) && OpenSSL::SSL::SSLError === err
raise SSLError, err
else
raise ConnectionFailed, err
end
end
end
rescue ::Timeout::Error => err
raise Timeout, err
end
private
def net_http_request(request)
http_req = Net::HTTPGenericRequest.new(
request.verb.to_s.upcase, # request method
!!request.body, # is there a request body
:head != request.verb, # is there a response body
request.url.request_uri, # request uri path
request.header, # request headers
)
if body = request.body_io
http_req.body_stream = body
end
http_req
end
def perform_request(http, request, res)
http.request(net_http_request(request)) do |http_res|
res.status_code = http_res.code.to_i
http_res.each_header do |key, value|
res.header[key] = value
end
if :get == request.verb
http_res.read_body { |chunk| res.receive_body(chunk) }
else
res.receive_body(http_res.body)
end
end
end
end
end
end