Module: Minisky::Requests

Includes:
Ruby2Compat
Included in:
Minisky
Defined in:
lib/minisky/requests.rb

Overview

This module contains most of the Minisky code for making HTTP requests and managing authentication tokens. The module is included into the Minisky API client class and you'll normally use it through that class, but you can also include it into your custom class if you want to implement the data storage differently than using a local YAML file as Minisky does.

Instance Attribute Summary collapse

Instance Method Summary collapse

Instance Attribute Details

#auto_manage_tokens ⇒ Boolean

Tells whether the library should manage the access & refresh tokens automatically for you (default: true if there is a user config).

If true, #check_access is called before each request to make sure that there is a fresh access token available; if false, you will need to call #log_in and #perform_token_refresh manually when needed.

Returns:

  • (Boolean) —

    whether to automatically manage access tokens



87
88
89
 
# File 'lib/minisky/requests.rb', line 87

def auto_manage_tokens
  instance_variable_defined?('@auto_manage_tokens') ? @auto_manage_tokens : (config != nil)
end

#default_progress ⇒ String? Also known as: progress

A character to print before each request in #fetch_all as a progress indicator. Can also be passed explicitly instead or overridden using the progress: parameter. Default is '.' when running inside IRB, and nil otherwise.

Returns:

  • (String, nil) 


51
52
53
 
# File 'lib/minisky/requests.rb', line 51

def default_progress
  @default_progress
end

#send_auth_headers ⇒ Boolean

Tells whether to set authentication headers automatically (default: true if there is a user config).

If false, you will need to pass auth: 'sometoken' explicitly to requests that require authentication.

Returns:

  • (Boolean) —

    whether to set authentication headers in requests



74
75
76
 
# File 'lib/minisky/requests.rb', line 74

def send_auth_headers
  instance_variable_defined?('@send_auth_headers') ? @send_auth_headers : (config != nil)
end

#stop_fetch_on_empty_page ⇒ Boolean

By default, when fetch_all receives a response with an empty page with no records but which includes a cursor for the next page, it keeps fetching until it receives a response with null cursor. This is more technically correct, but can cause problems with some non-compliant APIs, so you can set this option to stop fetching when an empty page is received.

Returns:

  • (Boolean) 


61
62
63
 
# File 'lib/minisky/requests.rb', line 61

def stop_fetch_on_empty_page
  @stop_fetch_on_empty_page
end

Instance Method Details

#access_token_expired? ⇒ Boolean

Attempts to parse the user's access token as JWT, extract the expiration date from the payload, and check if the token hasn't expired yet.

Returns:

  • (Boolean) —

    true if the token's expiration time is more than a minute away

Raises:

  • (AuthError) —

    if the token is not a valid JWT, or user is not logged in



445
446
447
448
449
450
451
452
453
454
455
456
457
 
# File 'lib/minisky/requests.rb', line 445

def access_token_expired?
  if user&.access_token.nil?
    raise AuthError, "No access token (user is not logged in)"
  end

  exp_date = token_expiration_date(user.access_token)

  if exp_date
    exp_date < Time.now + 60
  else
    raise AuthError, "Token expiration date cannot be decoded"
  end
end

#base_url ⇒ Object 



94
95
96
97
98
99
100
 
# File 'lib/minisky/requests.rb', line 94

def base_url
  if host.include?('://')
    host.chomp('/') + '/xrpc'
  else
    "https://#{host}/xrpc"
  end
end

#check_access ⇒ Symbol

Ensures that the user has a fresh access token, by checking the access token's expiry date and performing a refresh if needed, or by logging in with a password if no tokens are present.

If #auto_manage_tokens is enabled (the default setting), this method is automatically called before #get_request, #post_request and #fetch_all, so you generally don't need to call it yourself.

Returns:

  • (Symbol) —
    • :logged_in if a login using a password was performed
    • :refreshed if the access token was expired and was refreshed
    • :ok if no refresh was needed
    • :unknown if the token is not a valid JWT (e.g. an opaque blob)

Raises:

  • (BadResponse) —

    if login or refresh returns an error status code

  • (AuthError) —
    • if the client doesn't include user config at all
    • if logging in is required, but login or password isn't provided
    • if token refresh is needed, but refresh token is missing


325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
 
# File 'lib/minisky/requests.rb', line 325

def check_access
  if !user
    raise AuthError, "User config is missing"
  elsif !user.has_credentials?
    raise AuthError, "User id or password is missing"
  elsif !user.logged_in?
    
    return :logged_in
  end

  begin
    expired = access_token_expired?
  rescue AuthError
    return :unknown
  end

  if expired
    perform_token_refresh
    :refreshed
  else
    :ok
  end
end

#fetch_all(method, params = nil, auth: default_auth_mode, field: nil, break_when: nil, max_pages: nil, headers: nil, progress: @default_progress) ⇒ Array

Fetches and merges paginated responses from a service's endpoint in a loop, updating the cursor after each page, until the cursor is nil or a break condition is met. The data is extracted from a designated field of the response (field) and added to a single array, which is returned at the end.

A condition for when the fetching should stop can be passed as a block in break_when, or alternatively, a max number of pages can be passed to max_pages (or both together). If neither is set, the fetching continues until the server returns an empty cursor.

When experimenting in the Ruby console, you can pass nil as field (or skip the parameter) to make a single request and raise an exception, which will tell you what fields are available.

Examples:

Fetching with a break_when block

sky = Minisky.new('public.api.bsky.app', nil)
time_limit = Time.now - 86400 * 30

sky.fetch_all('app.bsky.feed.getAuthorFeed',
  { actor: 'pfrazee.com', limit: 100 },
  field: 'feed',
  progress: '|',
  break_when: ->(x) { Time.at(x['post']['record']['createdAt']) < time_limit }
)

Fetching with max_pages

sky = Minisky.new('tngl.sh', 'config.yml')
sky.fetch_all('app.bsky.feed.getTimeline', { limit: 100 }, field: 'feed', max_pages: 10)

Making a request in the console with empty field

sky = Minisky.new('public.api.bsky.app', nil)
# => #<Minisky:0x0000000120f5f6b0 @host="public.api.bsky.app", ...>

sky.fetch_all('app.bsky.graph.getFollowers', { actor: 'sdk.blue' })
# ./lib/minisky/requests.rb:270:in 'block in Minisky::Requests#fetch_all':
#   Field parameter not provided; available fields: ["followers"] (Minisky::FieldNotSetError)

sky.fetch_all('app.bsky.graph.getFollowers', { actor: 'sdk.blue' }, field: 'followers')
# => .....

Parameters:

  • method (String, URI) —

    an XRPC endpoint name or a full URL

  • params (Hash, nil) (defaults to: nil) —

    query parameters

  • auth (Boolean, String) (defaults to: default_auth_mode) —

    boolean value which tells whether to send an auth header with the access token or not, or an explicit bearer token to use

  • field (String, nil) (defaults to: nil) —

    name of the field in the responses which contains the data array

  • break_when (Proc, nil) (defaults to: nil) —

    if passed, the fetching will stop when the block returns true for any of the returned records, and records matching the condition will be deleted from the last page

  • max_pages (Integer, nil) (defaults to: nil) —

    maximum number of pages to fetch

  • headers (Hash, nil) (defaults to: nil) —

    additional headers to include

  • progress (String, nil) (defaults to: @default_progress) —

    a character to print before each request as a progress indicator

Returns:

  • (Array) —

    records or objects collected from all pages

Raises:

  • (ArgumentError) —

    if method name is invalid

  • (FieldNotSetError) —

    if field parameter wasn't set (the message tells you what fields were in the response)

  • (BadResponse) —

    if the HTTP response has an error status code

  • (AuthError) —
    • if logging in is required, but login or password isn't provided
    • if token refresh is needed, but refresh token is missing
    • if a token has invalid format
    • if required access token is missing, and #auto_manage_tokens is disabled


276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
 
# File 'lib/minisky/requests.rb', line 276

def fetch_all(method, params = nil, auth: default_auth_mode,
              field: nil, break_when: nil, max_pages: nil, headers: nil, progress: @default_progress)
  data = []
  params = {} if params.nil?
  pages = 0

  loop do
    print(progress) if progress

    response = get_request(method, params, auth: auth, headers: headers)

    if field.nil?
      raise FieldNotSetError, response.keys.select { |f| response[f].is_a?(Array) }
    end

    records = response[field]
    cursor = response['cursor']

    data.concat(records)
    params[:cursor] = cursor
    pages += 1

    break if !cursor || pages == max_pages || (stop_fetch_on_empty_page && records.empty?)
    break if break_when && records.any? { |x| break_when.call(x) }
  end

  data.delete_if { |x| break_when.call(x) } if break_when
  data
end

#get_request(method, params = nil, auth: default_auth_mode, headers: nil) ⇒ Hash, String

Sends a GET request to the service's API.

Examples:

Unauthenticated call

sky = Minisky.new('public.api.bsky.app', nil)
profile = sky.get_request('app.bsky.actor.getProfile', { actor: 'ec.europa.eu' })

Authenticated call

sky = Minisky.new('blacksky.app', 'config.yml')
feed = sky.get_request('app.bsky.feed.getTimeline', { limit: 100 })

Parameters:

  • method (String, URI) —

    an XRPC endpoint name or a full URL

  • params (Hash, nil) (defaults to: nil) —

    query parameters

  • auth (Boolean, String) (defaults to: default_auth_mode) —

    boolean value which tells whether to send an auth header with the access token or not, or an explicit bearer token to use

  • headers (Hash, nil) (defaults to: nil) —

    additional headers to include

Returns:

  • (Hash, String) —

    parsed JSON hash for JSON responses, or raw response body otherwise

Raises:

  • (ArgumentError) —

    if method name is invalid

  • (BadResponse) —

    if the HTTP response has an error status code

  • (AuthError) —
    • if logging in is required, but login or password isn't provided
    • if token refresh is needed, but refresh token is missing
    • if a token has invalid format
    • if required access token is missing, and #auto_manage_tokens is disabled


135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
 
# File 'lib/minisky/requests.rb', line 135

def get_request(method, params = nil, auth: default_auth_mode, headers: nil)
  check_access if auto_manage_tokens && auth == true

  headers = authentication_header(auth).merge(headers || {})
  url = build_request_uri(method)

  if params && !params.empty?
    url.query = URI.encode_www_form(params)
  end

  request = Net::HTTP::Get.new(url, headers)

  response = make_request(request)
  handle_response(response)
end

#log_in ⇒ Hash

Logs in the user using an ID and password stored in the config by calling the createSession endpoint, and stores the received access & refresh tokens.

This is generally handled automatically by #check_access. Calling this method repeatedly many times in a short period of time may use up your rate limit for this endpoint (which is lower than for others) and make it inaccessible to you for some time.

Returns:

  • (Hash) —

    the response JSON with access tokens

Raises:

  • (AuthError) —

    if login or password are missing

  • (BadResponse) —

    if the server responds with an error status code



362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
 
# File 'lib/minisky/requests.rb', line 362

def 
  if user.nil? || !user.has_credentials?
    raise AuthError, "To log in, please provide a user id and password"
  end

  data = {
    identifier: user.id,
    password: user.pass
  }

  if user.id =~ /\A[^@]+@[^@]+\z/
    STDERR.puts "Warning: logging in using an email address is deprecated in Minisky and will be " +
      "removed in a future version. Use either a handle or a DID instead."
  end

  json = post_request('com.atproto.server.createSession', data, auth: false)

  user.did = json['did']
  user.access_token = json['accessJwt']
  user.refresh_token = json['refreshJwt']

  save_config
  json
end

#perform_token_refresh ⇒ Hash

Refreshes the access token using the stored refresh token. If successful, this invalidates both old tokens and replaces them with new ones from the response.

If #auto_manage_tokens is enabled (the default setting), this method is automatically called before any requests through #check_access, so you generally don't need to call it yourself.

Returns:

  • (Hash) —

    the response JSON with access tokens

Raises:

  • (AuthError) —

    if the refresh token is missing

  • (BadResponse) —

    if the server responds with an error status code



398
399
400
401
402
403
404
405
406
407
408
409
410
 
# File 'lib/minisky/requests.rb', line 398

def perform_token_refresh
  if user&.refresh_token.nil?
    raise AuthError, "Can't refresh access token - refresh token is missing"
  end

  json = post_request('com.atproto.server.refreshSession', auth: user.refresh_token)

  user.access_token = json['accessJwt']
  user.refresh_token = json['refreshJwt']

  save_config
  json
end

#post_request(method, data = nil, auth: default_auth_mode, headers: nil, params: nil) ⇒ Hash, String

Sends a POST request to the service's API.

Examples:

Making a Bluesky post

sky = Minisky.new('lab.martianbase.net', 'config.yml')

sky.post_request('com.atproto.repo.createRecord', {
  repo: sky.user.did,
  collection: 'app.bsky.feed.post',
  record: {
    text: "Hello Bluesky!",
    createdAt: Time.now.iso8601,
    langs: ['en']
  }
})

Parameters:

  • method (String, URI) —

    an XRPC endpoint name or a full URL

  • data (Hash, String, nil) (defaults to: nil) —

    JSON or string data to send

  • auth (Boolean, String) (defaults to: default_auth_mode) —

    boolean value which tells whether to send an auth header with the access token or not, or an explicit bearer token to use

  • headers (Hash, nil) (defaults to: nil) —

    additional headers to include

  • params (Hash, nil) (defaults to: nil) —

    query parameters to append to the URL

Returns:

  • (Hash, String) —

    parsed JSON hash for JSON responses, or raw response body otherwise

Raises:

  • (ArgumentError) —

    if method name is invalid

  • (BadResponse) —

    if the HTTP response has an error status code

  • (AuthError) —
    • if logging in is required, but login or password isn't provided
    • if token refresh is needed, but refresh token is missing
    • if a token has invalid format
    • if required access token is missing, and #auto_manage_tokens is disabled


187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
 
# File 'lib/minisky/requests.rb', line 187

def post_request(method, data = nil, auth: default_auth_mode, headers: nil, params: nil)
  check_access if auto_manage_tokens && auth == true

  headers = authentication_header(auth).merge(headers || {})

  if data.is_a?(String) || data.nil?
    body = data.to_s
  else
    body = data.to_json
    headers["Content-Type"] = "application/json" unless headers.keys.any? { |k| k.to_s.downcase == 'content-type' }
  end

  url = build_request_uri(method)

  if params && !params.empty?
    url.query = URI.encode_www_form(params)
  end

  response = Net::HTTP.post(url, body, headers)
  handle_response(response)
end

#reset_tokens ⇒ Object

Clear stored access and refresh tokens, effectively logging out the user.

Raises:

  • (AuthError) —

    if the client doesn't have a user config



465
466
467
468
469
470
471
472
473
474
 
# File 'lib/minisky/requests.rb', line 465

def reset_tokens
  if !user
    raise AuthError, "User config is missing"
  end

  user.access_token = nil
  user.refresh_token = nil
  save_config
  nil
end

#token_expiration_date(token) ⇒ Time?

Attempts to parse a given token as JWT and extract the expiration date from the payload. An access token technically isn't required to be a (valid) JWT, so if the parsing fails for whatever reason, nil is returned.

Returns:

  • (Time, nil) —

    parsed expiration time, or nil if token is not a valid JWT



418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
 
# File 'lib/minisky/requests.rb', line 418

def token_expiration_date(token)
  return nil unless token.valid_encoding?

  parts = token.split('.')
  return nil unless parts.length == 3

  begin
    payload = JSON.parse(Base64.decode64(parts[1]))
  rescue JSON::ParserError
    return nil
  end

  exp = payload['exp']
  return nil unless exp.is_a?(Numeric) && exp > 0

  time = Time.at(exp)
  return nil if time.year < 2023 || time.year > 2100

  time
end

#user ⇒ Object 



102
103
104
 
# File 'lib/minisky/requests.rb', line 102

def user
  @user ||= config && User.new(config)
end