#RubyOnRails - 31 August 2018
« Back 1 day Forward 1 day »
[07:14:03] Applitect: What's the easiest way to merge mysql data from one set of tables to another set of tables while keeping any association ids intact?
[07:17:11] sevenseacat: I've written scripts to do once-off migrations in the past and they're always tricky
[07:18:00] baweaver: As with a lot of things it depends heavily on context and the actual task at hand
[07:18:09] Applitect: I've been writing some once-off merge Ruby scripts but it's getting pretty damn complicated.
[07:18:30] baweaver: ( though I swear one could get away with just repeatedly asking why or what questions and go on for at least 10 messages )
[07:19:03] baweaver: You have tables A, B, and C but want to move them to be the new tables D, E, and F?
[07:19:44] baweaver: because I swear the number of times a solution clicked before I pressed enter to ask here...
[07:21:18] Applitect: So I've taken over a project. I need to merge Beta into Production. Beta & Production share many tables in the same database. However, Beta has it's own certain set of tables used to separate data from Production. This "Beta" tables have a prefix of "beta_"
[07:22:14] Applitect: The tricky part is that the "beta_" tables contain earlier data from when they were duplicated from the Production tables.
[07:23:09] baweaver: Applitect: What's saved my sanity in similar migrations was to draw out both and literally draw mappings between the two, explaining the transformations and collisions.
[07:23:37] Applitect: I have to find a way to compare the "beta_" and Production tables... then merge all of the newer data into the Production tables while keeping any association Ids intact
[07:27:20] jane_booty_doe: i kind of just want to go off of this since i already use docker-compose https://github.com/ledermann/docker-rails
[07:27:24] baweaver: specify what each migration should look like from A to B, and test that before running it.
[07:27:32] Applitect: Instead of adding a "beta" column which holds a true / false... they chose to make "beta_" tables to keep the data separate.
[07:29:02] baweaver: Step 1) Don't let the person that made that call touch production data for a while
[07:29:23] baweaver: What I would do is qualify what makes a matching row. Start with the most central model you have (like User)
[07:29:39] baweaver: resolve that, then keep track of which beta_ rows migrated to which production ones
[07:30:02] sevenseacat: I usually start with the most peripheral model so you don't have to worry about associations just yet
[07:30:05] baweaver: So if you happen to find associations after reconciling dependencies you can map old beta ids to the proper new production ones
[07:30:06] Applitect: I've written a set of Ruby scripts to do the merge. The way I had to code it is highly inefficient. And... I've only tested them for maybe 100 or so entries in each table.
[07:30:13] Andr3as: This variant of the full outer join (sometimes abbreviated to full join) returns all rows from both tables named in the ON condition where the join condition is not met (excluding NULL values).
[07:31:22] Applitect: Because bascially... every time my scripts have to do a find for something like a Category... it then has to look through any other tables to find something that's associated with that category id
[07:31:25] Andr3as: This join is a variant on the basic right outer join, but instead, it returns all rows from the right-hand table specified in the ON condition that also meets the join condition and none of the rows from the left-hand table that match the join condition.
[07:33:20] Andr3as: here's a pic and stuff https://stackoverflow.com/questions/42553844/right-join-without-intersection-and-another-condition
[07:33:34] sevenseacat: eg. beta article 1 corresponds to beta category 2 - create new category record, note the ID mapping, then create new article with new category ID
[07:36:16] baweaver: There's going to be a beta and maybe a production value. Ideally the production value and id takes precedence I would guess.
[07:37:02] baweaver: The reason I mention starting with a central model is that it allows you to make a mapping of all the beta -> production id changes from that central model
[07:37:06] baweaver: Which allows you to, if you store it in a hash, not query against all of that data again
[07:47:09] Applitect: I'm start with something widely used like a category model first. I loop through the categories and see if they're present in both tables. IF they're not... I store it in a temp table and then proceed to find anything attached to it.
[07:47:33] Applitect: I think I'm just going to have to run my scripts as they are even though it's going to take ages.
[09:05:35] dionysus69: does this error make sense? I can't see why the division would cause that error, very unintuitive
[09:30:03] Ecco: Hi! Quick question, that is not strictly Rails-related: what tools do you guys recommend to generate "nice" HTML emails without going through tremendous pain?
[09:30:26] Ecco: I'd happily give up flexibility for ease-of-use (e.g. I don't mind if I can't customize everything)
[09:34:04] Ecco: Seems like an awful lot of trouble. How can you tell it works well given across the different clients?
[09:36:22] Sylario: Work on the typo settings, put a nice background color, it's a minimalism exercise
[11:47:23] jlebrech: I can't find a way to make integration tests to test angularjs code on ruby. what do you use? teaspoon?
[16:17:22] machinewar: why would same ruby version (2.5.1) have different results for "irb" v. "rails console" Rack::Utils.escape_path(":"). rails console URL escapes the value while irb doesnt. Does rails monkey patch rack::utils?
[16:19:31] machinewar: or more likely my ruby installed version is different than my rails projects.....
[16:59:17] ShekharReddy: Hi guys, I am abit novice to rails.. a quick question -- are params accessible from a method in a module in controller directory (I am invoking that method which is in the module from a controller action method )