Class: SuttyMigration::Wordpress

Inherits:
Object
  • Object
show all
Defined in:
lib/sutty_migration/wordpress.rb

Overview

Brings posts and attachments from a SQLite3 database. You can convert a MySQL/MariaDB dump by using ‘mysql2sqlite`.

It doesn’t convert them into Jekyll posts but allows you to write a migration plugin where you can convert data by yourself. We may add this feature in the future.

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(site:, url:, database: nil, prefix: 'wp_', limit: 10, multisite: nil) ⇒ Wordpress

Returns a new instance of Wordpress.

Parameters:

  • :site (Jekyll::Site) —

    Jekyll site

  • :url (String) —

    Wordpress site URL (must be up for downloads)

  • :database (String) —

    Database path, by default ‘_data/wordpress.sqlite3`

  • :prefix (String) —

    WP table prefix

  • :limit (Integer) —

    Page length

  • :multisite (Boolean) —

    Site is multisite



28
29
30
31
32
33
34
35
 
# File 'lib/sutty_migration/wordpress.rb', line 28

def initialize(site:, url:, database: nil, prefix: 'wp_', limit: 10, multisite: nil)
  @site = site
  @prefix = prefix.freeze
  @limit = limit.freeze
  @url = url.freeze
  @database = database || File.join(site.source, '_data', 'wordpress.sqlite3')
  @multisite = multisite
end

Instance Attribute Details

#database ⇒ Object (readonly)

Returns the value of attribute database.



20
21
22
 
# File 'lib/sutty_migration/wordpress.rb', line 20

def database
  @database
end

#limit ⇒ Object (readonly)

Returns the value of attribute limit.



20
21
22
 
# File 'lib/sutty_migration/wordpress.rb', line 20

def limit
  @limit
end

#multisite ⇒ Object (readonly)

Returns the value of attribute multisite.



20
21
22
 
# File 'lib/sutty_migration/wordpress.rb', line 20

def multisite
  @multisite
end

#prefix ⇒ Object (readonly)

Returns the value of attribute prefix.



20
21
22
 
# File 'lib/sutty_migration/wordpress.rb', line 20

def prefix
  @prefix
end

#site ⇒ Object (readonly)

Returns the value of attribute site.



20
21
22
 
# File 'lib/sutty_migration/wordpress.rb', line 20

def site
  @site
end

#url ⇒ Object (readonly)

Returns the value of attribute url.



20
21
22
 
# File 'lib/sutty_migration/wordpress.rb', line 20

def url
  @url
end

#wp ⇒ Sequel::SQLite::Database (readonly)

Open the database.

Returns:

  • (Sequel::SQLite::Database) 


59
60
61
 
# File 'lib/sutty_migration/wordpress.rb', line 59

def wp
  @wp
end

Instance Method Details

#blogs ⇒ Hash

Generate database connections for a multisite WP

Returns:

  • (Hash) —

    { “ID” => SuttyMigration::Wordpress }



40
41
42
43
44
45
46
47
48
49
50
 
# File 'lib/sutty_migration/wordpress.rb', line 40

def blogs
  @blogs ||= wp["select * from #{prefix}blogs"].to_a.map do |blog|
    url   = "https://#{blog[:domain]}#{blog[:path]}"
    pfx   = "#{prefix}#{blog[:blog_id]}_" if blog[:blog_id] > 1
    pfx ||= prefix

    [blog[:blog_id],
     blog.merge(db: self.class.new(site: site, url: url, prefix: pfx, database: database, limit: limit,
                                   multisite: self))]
  end.to_h
end

#download(url:, progress: true) ⇒ String

Downloads a file if needed, optionally showing a progress bar.

Parameters:

  • :url (String) —

    File URL

  • :progress (Boolean) —

    Toggle progress bar

Returns:

  • (String) —

    File local path



80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
 
# File 'lib/sutty_migration/wordpress.rb', line 80

def download(url:, progress: true)
  uri = URI(url)
  dest = uri.path.sub(%r{\A/}, '')
  full = File.join(site.source, dest)

  return dest if File.exist? full

  ::Jekyll.logger.info "Downloading #{dest}"

  FileUtils.mkdir_p File.dirname(full)

  File.open(full, 'w') do |f|
    if progress
      head = Faraday.head(url)
      content_length = head.headers['content-length'].to_i
      progress = ProgressBar.create(title: File.basename(dest), total: content_length, output: $stderr)
    end

    Faraday.get(url) do |req|
      req.options.on_data = proc do |chunk, downloaded_bytes|
        f.write chunk

        if progress
          progress.progress = downloaded_bytes > content_length ? content_length : downloaded_bytes
        end
      end
    end
  end

  dest
end

#download_all(progress: true) ⇒ Nil

Download all attachments. Adds the local path to them.

Parameters:

  • :progress (Boolean) —

    Toggle progress bar

Returns:

  • (Nil) 


69
70
71
72
73
 
# File 'lib/sutty_migration/wordpress.rb', line 69

def download_all(progress: true)
  posts(layout: 'attachment').each do |attachment|
    attachment[:front_matter]['file_path'] = download(url: attachment[:guid], progress: progress)
  end
end

#layouts ⇒ Array

List post types

Returns:

  • (Array) 


115
116
117
 
# File 'lib/sutty_migration/wordpress.rb', line 115

def layouts
  @layouts ||= wp["select distinct post_type from #{prefix}posts"].to_a.map(&:values).flatten
end

#options ⇒ Object 



52
53
54
 
# File 'lib/sutty_migration/wordpress.rb', line 52

def options
  @options ||= wp["select option_name, option_value from #{prefix}options"].to_a.map(&:values).to_h.transform_keys(&:to_sym)
end

#posts(**options) ⇒ Enumerator

Finds all posts optionally filtering by post type. This is not the official Sequel syntax, but it retrieves metadata as objects with a single query (and a sub-query).

Parameters:

  • :layout (String) —

    Layout name, one of #layouts

  • :with_meta (Boolean) —

    Toggle metadata pulling and conversion

Returns:

  • (Enumerator) 


126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
 
# File 'lib/sutty_migration/wordpress.rb', line 126

def posts(**options)
  unless options[:layout].blank? || layouts.include?(options[:layout])
    raise ArgumentError, "#{options[:layout]} must be one of #{layouts.join(', ')}"
  end

  wp[post_query(**options)].each_page(limit).to_a.map(&:to_a).flatten.tap do |p|
    p.map do |post|
      # Sequel parses dates on localtime
      post[:date] = ::Jekyll::Utils.parse_date(post[:date]) unless post[:date].blank?
      unless post[:last_modified_at].blank?
        post[:last_modified_at] =
          ::Jekyll::Utils.parse_date(post[:last_modified_at])
      end

      post[:front_matter] =
        begin
          unless post[:front_matter].blank?
            JSON.parse(post[:front_matter]).transform_keys(&:to_sym).transform_values do |v|
              v.size == 1 ? v.first : v
            end
          end
        rescue JSON::ParserError
          {}
        end
      post[:terms] =
        begin
          unless post[:terms].blank?
            JSON.parse(post[:terms]).transform_keys(&:to_sym).transform_values do |v|
              v.size == 1 ? v.first : v
            end
          end
        rescue JSON::ParserError
          {}
        end
    end
  end
end

#users(**options) ⇒ Array

Brings all users.

Parameters:

  • :with_meta (Boolean) —

    include metadata

Returns:

  • (Array) 


168
169
170
171
172
173
174
175
176
177
178
 
# File 'lib/sutty_migration/wordpress.rb', line 168

def users(**options)
  options[:with_meta] = true unless options.key? :with_meta

  wp[user_query(**options)].each_page(limit).to_a.map(&:to_a).flatten.tap do |u|
    next unless options[:with_meta]

    u.map do |user|
      user[:meta] = JSON.parse(user[:meta]).transform_keys(&:to_sym) unless user[:meta].blank?
    end
  end
end