Class: JSON::LD::API
- Inherits:
-
Object
- Object
- JSON::LD::API
- Defined in:
- lib/json/ld/api.rb,
lib/json/ld/html/rexml.rb,
lib/json/ld/html/nokogiri.rb
Overview
A JSON-LD processor based on the JsonLdProcessor interface.
This API provides a clean mechanism that enables developers to convert JSON-LD data into a a variety of output formats that are easier to work with in various programming languages. If a JSON-LD API is provided in a programming environment, the entirety of the following API must be implemented.
Note that the API method signatures are somewhat different than what is specified, as the use of Futures and explicit callback parameters is not as relevant for Ruby-based interfaces.
Defined Under Namespace
Modules: Nokogiri, REXML Classes: RemoteDocument
Constant Summary collapse
- OPEN_OPTS =
Options used for open_file
{ headers: {"Accept" => "application/ld+json, text/html;q=0.8, application/xhtml+xml;q=0.8, application/json;q=0.5"} }
- LINK_REL_CONTEXT =
The following constants are used to reduce object allocations
%w(rel http://www.w3.org/ns/json-ld#context).freeze
- LINK_REL_ALTERNATE =
%w(rel alternate).freeze
- LINK_TYPE_JSONLD =
%w(type application/ld+json).freeze
- JSON_LD_PROCESSING_MODES =
%w(json-ld-1.0 json-ld-1.1).freeze
Constants included from Compact
Compact::CONTAINER_MAPPING_LANGUAGE_INDEX_ID_TYPE, Compact::EXPANDED_PROPERTY_DIRECTION_INDEX_LANGUAGE_VALUE
Constants included from Expand
Expand::CONTAINER_INDEX_ID_TYPE, Expand::KEYS_INCLUDED_TYPE, Expand::KEYS_SET_LIST_INDEX, Expand::KEYS_VALUE_LANGUAGE_TYPE_INDEX_DIRECTION, Expand::KEY_ID
Instance Attribute Summary collapse
-
#context ⇒ JSON::LD::Context
Input evaluation context.
-
#input ⇒ String, ...
Current input.
-
#namer ⇒ JSON::LD::BlankNodeNamer
readonly
Current Blank Node Namer.
-
#value ⇒ String, ...
Current input.
Class Method Summary collapse
-
.compact(input, context, expanded: false, **options) {|jsonld| ... } ⇒ Object, Hash
Compacts the given input according to the steps in the Compaction Algorithm.
-
.documentLoader(url, extractAllScripts: false, profile: nil, requestProfile: nil, **options) {|remote_document| ... } ⇒ Object
Default document loader.
-
.expand(input, framing: false, **options) {|jsonld, base_iri| ... } ⇒ Object, Array<Hash>
Expands the given input according to the steps in the Expansion Algorithm.
-
.flatten(input, context, expanded: false, **options) {|jsonld| ... } ⇒ Object, Hash
This algorithm flattens an expanded JSON-LD document by collecting all properties of a node in a single JSON object and labeling all blank nodes with blank node identifiers.
-
.frame(input, frame, expanded: false, **options) {|jsonld| ... } ⇒ Object, Hash
Frames the given input using the frame according to the steps in the Framing Algorithm.
-
.fromRdf(input, useRdfType: false, useNativeTypes: false, **options) {|jsonld| ... } ⇒ Object, Hash
(also: fromRDF)
Take an ordered list of RDF::Statements and turn them into a JSON-LD document.
-
.load_html(input, url:, library: nil, extractAllScripts: false, profile: nil, **options) ⇒ Object
Load one or more script tags from an HTML source.
-
.loadRemoteDocument(url, extractAllScripts: false, profile: nil, requestProfile: nil, validate: false, base: nil, **options) {|remote_document| ... } ⇒ Object, RemoteDocument
Uses built-in or provided documentLoader to retrieve a parsed document.
-
.toRdf(input, expanded: false, **options) {|statement| ... } ⇒ RDF::Enumerable
(also: toRDF)
Processes the input according to the RDF Conversion Algorithm, calling the provided callback for each triple generated.
Methods included from Frame
#cleanup_null, #cleanup_preserve, #count_blank_node_identifiers, #count_blank_node_identifiers_internal, #frame, #prune_bnodes, #remove_dependents
Methods included from Utils
#add_value, #as_array, #as_resource, #blank_node?, #compare_values, #graph?, #has_property, #has_value, #index?, #list?, #node?, #node_or_ref?, #node_reference?, #simple_graph?, #value?
Methods included from FromRDF
Methods included from Flatten
Methods included from ToRDF
#item_to_rdf, #node, #parse_list
Methods included from Compact
Methods included from Expand
Instance Attribute Details
#context ⇒ JSON::LD::Context
Input evaluation context
55 56 57 |
# File 'lib/json/ld/api.rb', line 55 def context @context end |
#input ⇒ String, ...
Current input
50 |
# File 'lib/json/ld/api.rb', line 50 attr_accessor :value |
#namer ⇒ JSON::LD::BlankNodeNamer (readonly)
Current Blank Node Namer
60 61 62 |
# File 'lib/json/ld/api.rb', line 60 def namer @namer end |
#value ⇒ String, ...
Current input
50 51 52 |
# File 'lib/json/ld/api.rb', line 50 def value @value end |
Class Method Details
.compact(input, context, expanded: false, **options) {|jsonld| ... } ⇒ Object, Hash
Compacts the given input according to the steps in the Compaction Algorithm. The input must be copied, compacted and returned if there are no errors. If the compaction fails, an appropirate exception must be thrown.
If no context is provided, the input document is compacted using the top-level context of the document
The resulting ‘Hash` is either returned or yielded, if a block is given.
214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 |
# File 'lib/json/ld/api.rb', line 214 def self.compact(input, context, expanded: false, **) result = nil = {compactToRelative: true}.merge() # 1) Perform the Expansion Algorithm on the JSON-LD input. # This removes any existing context to allow the given context to be cleanly applied. = ? input : API.(input, ordered: false, **) do |res, base_iri| [:base] ||= base_iri if [:compactToRelative] res end API.new(, context, no_default_base: true, **) do log_debug(".compact") {"expanded input: #{.to_json(JSON_STATE) rescue 'malformed json'}"} result = compact(value, ordered: @options[:ordered]) # xxx) Add the given context to the output ctx = self.context.serialize if result.is_a?(Array) kwgraph = self.context.compact_iri('@graph', vocab: true) result = result.empty? ? {} : {kwgraph => result} end result = ctx.merge(result) unless ctx.empty? end block_given? ? yield(result) : result end |
.documentLoader(url, extractAllScripts: false, profile: nil, requestProfile: nil, **options) {|remote_document| ... } ⇒ Object
Default document loader.
652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 |
# File 'lib/json/ld/api.rb', line 652 def self.documentLoader(url, extractAllScripts: false, profile: nil, requestProfile: nil, **, &block) case url when IO, StringIO base_uri = [:base] base_uri ||= url.base_uri if url.respond_to?(:base_uri) content_type = [:content_type] content_type ||= url.content_type if url.respond_to?(:content_type) context_url = if url.respond_to?(:links) && url.links (content_type == 'appliaction/json' || content_type.match?(%r(application/(^ld)+json))) link = url.links.find_link(LINK_REL_CONTEXT) link.href if link end block.call(RemoteDocument.new(url.read, documentUrl: base_uri, contentType: content_type, contextUrl: context_url)) else RDF::Util::File.open_file(url, **, &block) end end |
.expand(input, framing: false, **options) {|jsonld, base_iri| ... } ⇒ Object, Array<Hash>
Expands the given input according to the steps in the Expansion Algorithm. The input must be copied, expanded and returned if there are no errors. If the expansion fails, an appropriate exception must be thrown.
The resulting ‘Array` either returned or yielded
165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 |
# File 'lib/json/ld/api.rb', line 165 def self.(input, framing: false, **, &block) result, doc_base = nil API.new(input, [:expandContext], **) do result = self.(self.value, nil, self.context, ordered: @options[:ordered], framing: framing) doc_base = @options[:base] end # If, after the algorithm outlined above is run, the resulting element is an JSON object with just a @graph property, element is set to the value of @graph's value. result = result['@graph'] if result.is_a?(Hash) && result.length == 1 && result.key?('@graph') # Finally, if element is a JSON object, it is wrapped into an array. result = [result].compact unless result.is_a?(Array) if block_given? case block.arity when 1 then yield(result) when 2 then yield(result, doc_base) else raise "Unexpected number of yield parameters to expand" end else result end end |
.flatten(input, context, expanded: false, **options) {|jsonld| ... } ⇒ Object, Hash
This algorithm flattens an expanded JSON-LD document by collecting all properties of a node in a single JSON object and labeling all blank nodes with blank node identifiers. This resulting uniform shape of the document, may drastically simplify the code required to process JSON-LD data in certain applications.
The resulting ‘Array` is either returned, or yielded if a block is given.
259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 |
# File 'lib/json/ld/api.rb', line 259 def self.flatten(input, context, expanded: false, **) flattened = [] = { compactToRelative: true, extractAllScripts: true, }.merge() # Expand input to simplify processing = ? input : API.(input, **) do |result, base_iri| [:base] ||= base_iri if [:compactToRelative] result end # Initialize input using API.new(, context, no_default_base: true, **) do log_debug(".flatten") {"expanded input: #{value.to_json(JSON_STATE) rescue 'malformed json'}"} # Initialize node map to a JSON object consisting of a single member whose key is @default and whose value is an empty JSON object. graph_maps = {'@default' => {}} create_node_map(value, graph_maps) default_graph = graph_maps['@default'] graph_maps.keys.opt_sort(ordered: @options[:ordered]).each do |graph_name| next if graph_name == '@default' graph = graph_maps[graph_name] entry = default_graph[graph_name] ||= {'@id' => graph_name} nodes = entry['@graph'] ||= [] graph.keys.opt_sort(ordered: @options[:ordered]).each do |id| nodes << graph[id] unless node_reference?(graph[id]) end end default_graph.keys.opt_sort(ordered: @options[:ordered]).each do |id| flattened << default_graph[id] unless node_reference?(default_graph[id]) end if context && !flattened.empty? # Otherwise, return the result of compacting flattened according the Compaction algorithm passing context ensuring that the compaction result uses the @graph keyword (or its alias) at the top-level, even if the context is empty or if there is only one element to put in the @graph array. This ensures that the returned document has a deterministic structure. compacted = as_array(compact(flattened, ordered: @options[:ordered])) kwgraph = self.context.compact_iri('@graph') flattened = self.context.serialize.merge(kwgraph => compacted) end end block_given? ? yield(flattened) : flattened end |
.frame(input, frame, expanded: false, **options) {|jsonld| ... } ⇒ Object, Hash
Frames the given input using the frame according to the steps in the Framing Algorithm. The input is used to build the framed output and is returned if there are no errors. If there are no matches for the frame, null must be returned. Exceptions must be thrown if there are errors.
The resulting ‘Array` is either returned, or yielded if a block is given.
335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 |
# File 'lib/json/ld/api.rb', line 335 def self.frame(input, frame, expanded: false, **) result = nil = { base: (input if input.is_a?(String)), compactArrays: true, compactToRelative: true, embed: '@once', explicit: false, requireAll: false, omitDefault: false, }.merge() framing_state = { graphMap: {}, graphStack: [], subjectStack: [], link: {}, embedded: false # False at the top-level } # de-reference frame to create the framing object frame = case frame when Hash then frame.dup when IO, StringIO, String remote_doc = loadRemoteDocument(frame, profile: 'http://www.w3.org/ns/json-ld#frame', requestProfile: 'http://www.w3.org/ns/json-ld#frame', **) if remote_doc.document.is_a?(String) MultiJson.load(remote_doc.document) else remote_doc.document end end # Expand input to simplify processing = ? input : API.(input, ordered: false, **) do |res, base_iri| [:base] ||= base_iri if [:compactToRelative] res end # Expand frame to simplify processing = API.(frame, framing: true, ordered: false, **) # Initialize input using frame as context API.new(, frame['@context'], no_default_base: true, **) do log_debug(".frame") {"expanded input: #{.to_json(JSON_STATE) rescue 'malformed json'}"} log_debug(".frame") {"expanded frame: #{.to_json(JSON_STATE) rescue 'malformed json'}"} if %w(@first @last).include?([:embed]) && context.processingMode('json-ld-1.1') raise JSON::LD::JsonLdError::InvalidEmbedValue, "#{[:embed]} is not a valid value of @embed in 1.1 mode" if @options[:validate] warn "[DEPRECATION] #{[:embed]} is not a valid value of @embed in 1.1 mode.\n" end # Set omitGraph option, if not present, based on processingMode unless .has_key?(:omitGraph) [:omitGraph] = context.processingMode('json-ld-1.1') end # Get framing nodes from expanded input, replacing Blank Node identifiers as necessary create_node_map(value, framing_state[:graphMap], active_graph: '@default') frame_keys = frame.keys.map {|k| context.(k, vocab: true)} if frame_keys.include?('@graph') # If frame contains @graph, it matches the default graph. framing_state[:graph] = '@default' else # If frame does not contain @graph used the merged graph. framing_state[:graph] = '@merged' framing_state[:link]['@merged'] = {} framing_state[:graphMap]['@merged'] = merge_node_map_graphs(framing_state[:graphMap]) end framing_state[:subjects] = framing_state[:graphMap][framing_state[:graph]] result = [] frame(framing_state, framing_state[:subjects].keys.opt_sort(ordered: @options[:ordered]), (.first || {}), parent: result, **) # Default to based on processinMode if !.has_key?(:pruneBlankNodeIdentifiers) [:pruneBlankNodeIdentifiers] = context.processingMode('json-ld-1.1') end # Count blank node identifiers used in the document, if pruning if [:pruneBlankNodeIdentifiers] bnodes_to_clear = count_blank_node_identifiers(result).collect {|k, v| k if v == 1}.compact result = prune_bnodes(result, bnodes_to_clear) end # Replace values with `@preserve` with the content of its entry. result = cleanup_preserve(result) log_debug(".frame") {"expanded result: #{result.to_json(JSON_STATE) rescue 'malformed json'}"} # Compact result compacted = compact(result, ordered: @options[:ordered]) # @replace `@null` with nil, compacting arrays compacted = cleanup_null(compacted) compacted = [compacted] unless [:omitGraph] || compacted.is_a?(Array) # Add the given context to the output result = if !compacted.is_a?(Array) context.serialize.merge(compacted) else kwgraph = context.compact_iri('@graph') context.serialize.merge({kwgraph => compacted}) end log_debug(".frame") {"after compact: #{result.to_json(JSON_STATE) rescue 'malformed json'}"} result end block_given? ? yield(result) : result end |
.fromRdf(input, useRdfType: false, useNativeTypes: false, **options) {|jsonld| ... } ⇒ Object, Hash Also known as: fromRDF
Take an ordered list of RDF::Statements and turn them into a JSON-LD document.
The resulting ‘Array` is either returned or yielded, if a block is given.
518 519 520 521 522 523 524 525 526 527 528 529 |
# File 'lib/json/ld/api.rb', line 518 def self.fromRdf(input, useRdfType: false, useNativeTypes: false, **, &block) result = nil API.new(nil, nil, **) do result = from_statements(input, useRdfType: useRdfType, useNativeTypes: useNativeTypes, ordered: @options[:ordered]) end block_given? ? yield(result) : result end |
.load_html(input, url:, library: nil, extractAllScripts: false, profile: nil, **options) ⇒ Object
Load one or more script tags from an HTML source. Unescapes and uncomments input, returns the internal representation Yields document base
690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 |
# File 'lib/json/ld/api.rb', line 690 def self.load_html(input, url:, library: nil, extractAllScripts: false, profile: nil, **) if input.is_a?(String) library ||= begin require 'nokogiri' :nokogiri rescue LoadError :rexml end require "json/ld/html/#{library}" # Parse HTML using the appropriate library implementation = case library when :nokogiri then Nokogiri when :rexml then REXML end self.extend(implementation) input = begin self.send("initialize_html_#{library}".to_sym, input, **) rescue raise JSON::LD::JsonLdError::LoadingDocumentFailed, "Malformed HTML document: #{$!.}" end # Potentially update options[:base] if html_base = input.at_xpath("/html/head/base/@href") base = RDF::URI(url) if url html_base = RDF::URI(html_base) html_base = base.join(html_base) if base yield html_base end end url = RDF::URI.parse(url) if url.fragment id = CGI.unescape(url.fragment) # Find script with an ID based on that fragment. element = input.at_xpath("//script[@id='#{id}']") raise JSON::LD::JsonLdError::LoadingDocumentFailed, "No script tag found with id=#{id}" unless element raise JSON::LD::JsonLdError::LoadingDocumentFailed, "Script tag has type=#{element.attributes['type']}" unless element.attributes['type'].to_s.start_with?('application/ld+json') content = element.inner_html validate_input(content, url: url) if [:validate] MultiJson.load(content, **) elsif extractAllScripts res = [] elements = if profile es = input.xpath("//script[starts-with(@type, 'application/ld+json;profile=#{profile}')]") # If no profile script, just take a single script without profile es = [input.at_xpath("//script[starts-with(@type, 'application/ld+json')]")].compact if es.empty? es else input.xpath("//script[starts-with(@type, 'application/ld+json')]") end elements.each do |element| content = element.inner_html validate_input(content, url: url) if [:validate] r = MultiJson.load(content, **) if r.is_a?(Hash) res << r elsif r.is_a?(Array) res = res.concat(r) end end res else # Find the first script with type application/ld+json. element = input.at_xpath("//script[starts-with(@type, 'application/ld+json;profile=#{profile}')]") if profile element ||= input.at_xpath("//script[starts-with(@type, 'application/ld+json')]") raise JSON::LD::JsonLdError::LoadingDocumentFailed, "No script tag found" unless element content = element.inner_html validate_input(content, url: url) if [:validate] MultiJson.load(content, **) end rescue MultiJson::ParseError => e raise JSON::LD::JsonLdError::InvalidScriptElement, e. end |
.loadRemoteDocument(url, extractAllScripts: false, profile: nil, requestProfile: nil, validate: false, base: nil, **options) {|remote_document| ... } ⇒ Object, RemoteDocument
Uses built-in or provided documentLoader to retrieve a parsed document.
552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 |
# File 'lib/json/ld/api.rb', line 552 def self.loadRemoteDocument(url, extractAllScripts: false, profile: nil, requestProfile: nil, validate: false, base: nil, **) documentLoader = .fetch(:documentLoader, self.method(:documentLoader)) = OPEN_OPTS.merge() if requestProfile # Add any request profile [:headers]['Accept'] = [:headers]['Accept'].sub('application/ld+json,', "application/ld+json;profile=#{requestProfile}, application/ld+json;q=0.9,") end documentLoader.call(url, **) do |remote_doc| case remote_doc when RDF::Util::File::RemoteDocument # Convert to RemoteDocument context_url = if remote_doc.content_type != 'application/ld+json' && (remote_doc.content_type == 'application/json' || remote_doc.content_type.to_s.match?(%r(application/\w+\+json))) # Get context link(s) # Note, we can't simply use #find_link, as we need to detect multiple links = remote_doc.links.links.select do |link| link.attr_pairs.include?(LINK_REL_CONTEXT) end raise JSON::LD::JsonLdError::MultipleContextLinkHeaders, "expected at most 1 Link header with rel=jsonld:context, got #{links.length}" if links.length > 1 Array(links.first).first end # If content-type is not application/ld+json, nor any other +json and a link with rel=alternate and type='application/ld+json' is found, use that instead alternate = !remote_doc.content_type.match?(%r(application/(\w*\+)?json)) && remote_doc.links.links.detect do |link| link.attr_pairs.include?(LINK_REL_ALTERNATE) && link.attr_pairs.include?(LINK_TYPE_JSONLD) end remote_doc = if alternate # Load alternate relative to URL loadRemoteDocument(RDF::URI(url).join(alternate.href), extractAllScripts: extractAllScripts, profile: profile, requestProfile: requestProfile, validate: validate, base: base, **) else RemoteDocument.new(remote_doc.read, documentUrl: remote_doc.base_uri, contentType: remote_doc.content_type, contextUrl: context_url) end when RemoteDocument # Pass through else raise JSON::LD::JsonLdError::LoadingDocumentFailed, "unknown result from documentLoader: #{remote_doc.class}" end # Use specified document location remote_doc.documentUrl = base if base # Parse any HTML if remote_doc.document.is_a?(String) remote_doc.document = case remote_doc.contentType when 'text/html', 'application/xhtml+xml' load_html(remote_doc.document, url: remote_doc.documentUrl, extractAllScripts: extractAllScripts, profile: profile, **) do |base| remote_doc.documentUrl = base end else validate_input(remote_doc.document, url: remote_doc.documentUrl) if validate MultiJson.load(remote_doc.document, **) end end if remote_doc.contentType && validate raise IOError, "url: #{url}, contentType: #{remote_doc.contentType}" unless remote_doc.contentType.match?(/application\/(.+\+)?json|text\/html|application\/xhtml\+xml/) end block_given? ? yield(remote_doc) : remote_doc end rescue IOError, MultiJson::ParseError => e raise JSON::LD::JsonLdError::LoadingDocumentFailed, e. end |
.toRdf(input, expanded: false, **options) {|statement| ... } ⇒ RDF::Enumerable Also known as: toRDF
Processes the input according to the RDF Conversion Algorithm, calling the provided callback for each triple generated.
462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 |
# File 'lib/json/ld/api.rb', line 462 def self.toRdf(input, expanded: false, **, &block) unless block_given? results = [] results.extend(RDF::Enumerable) self.toRdf(input, **) do |stmt| results << stmt end return results end = { extractAllScripts: true, }.merge() # Expand input to simplify processing = ? input : API.(input, ordered: false, **) API.new(, nil, **) do # 1) Perform the Expansion Algorithm on the JSON-LD input. # This removes any existing context to allow the given context to be cleanly applied. log_debug(".toRdf") {"expanded input: #{.to_json(JSON_STATE) rescue 'malformed json'}"} # Recurse through input .each do |node| item_to_rdf(node) do |statement| next if statement.predicate.node? && ![:produceGeneralizedRdf] # Drop invalid statements (other than IRIs) unless statement.valid_extended? log_debug(".toRdf") {"drop invalid statement: #{statement.to_nquads}"} next end yield statement end end end end |