#ruby - 05 November 2017
« Back 1 day Forward 1 day »
[00:06:44] apeiros: and no, there's no built-in delta comparison. mostly because the delta in delta comparisons is case sensitive. there's no silver-bullet delta value.
[00:07:09] RickHull: it's pretty annoying to have to confirm a straightforward result with the inverse operation
[00:11:15] RickHull: i am prioritizing cleanliness of expression, where confirming with the inverse is not clean
[00:11:19] apeiros: but if you care about speed, this kind of thing is at least relatively simple to translate into a C ext
[00:13:08] RickHull: kind of embarrassing. i want to print a binary tree, breadth first, with the root node in the first line
[00:15:58] RickHull: given e.g. width 80, the first line should be like 35 spaces and then the root val
[00:19:08] RickHull: yeah, and/or general progress through the tree. for example, count all of the children slots at a given level
[00:25:04] RickHull: to circle back, when I am looking at integers, it's usually on a C-type boundary -- int4, int8, int16
[00:26:25] apeiros: so you either have infinite leading 1's for negative numbers, or a specific number of bits for positive numbers.
[00:27:04] RickHull: if you have the bit string, on some byte boundary, you can just count the on bits. on_count == 1
[00:34:00] RickHull: here's the last time I used pack in anger: https://github.com/rickhull/device_input/blob/master/lib/device_input.rb
[00:57:02] nofxx: apeiros, kidding about the regex of course, but prime thing is one of them.. let me find
[00:57:18] elomatreb: You mean prime factorization, which is different from telling if a number is prime
[00:59:35] elomatreb: You mean the Riemann one or the P vs. NP one? Because both are sort-of relevant to this
[01:02:23] nofxx: Sistema Operacional in latin based langs, english and preceding adjetives, blergh... =P
[01:03:06] RickHull: usually things are developed on a desktop type machine, even if android is the target platform
[01:04:07] RickHull: I wonder what is the most straightforward way to get e.g. irb on an android device
[01:04:41] nofxx: RickHull, I remember jailbreaking the iphone 2 or 3 just to irb.. never did anything with it
[01:06:54] RickHull: aside: I want to embed a cheap rpi / arm thing in my mechanical keyboard. and use a phone or tablet as a display
[01:09:20] nofxx: wouldn't trust that, hm...and you'll need another device problably ( at least to work with common lcd's)
[01:12:48] RickHull: and for a train/bus, look into a laptop. chromebook or something running android-like can work, maybe
[01:23:23] RickHull: nofxx: a lot of the arm socs include HDMI, as of several years ago. I haven't paid much attention since
[01:53:23] RickHull: apeiros: https://github.com/rickhull/compsci/blob/master/lib/compsci/tree.rb#L41 :)
[01:53:53] RickHull: also, my assertion count and assertions/sec went through the roof: https://github.com/rickhull/compsci/blob/master/test/tree.rb#L67
[02:11:13] Swat2: salutations all... I'm wondering what the best way would be to write a function that has to pull a heap of rows out of a MSSQL database that has images stored, to save them as a physical image on the system with the image_id as it's filename. The data size could be quite large - maybe too big to fit in memory but constantly re-querying the database for each row appears to be slow and cumbersome.
[02:15:28] RickHull: images -- you mean a binary blob that represents the file contents of e.g. a photo?
[02:18:01] Swat2: mmm problem is ... MSSQL is on one server, and where the result needs to be stored is far far away.
[02:18:25] RickHull: if you are transferring binary blobs over the network, then that's probably the bottleneck
[02:19:23] RickHull: if you're IO bound, due to the simple magnitude of bits on the wire, it won't matter much how you get them
[02:20:20] RickHull: you might try the stored proc approach, write say 100 files to the db filesystem
[02:20:32] elomatreb: Naive way to deal with the fits-in-memory problem would be to page the result (LIMIT+OFFSET) into chunks that do fit
[02:21:00] RickHull: then upload those files to S3 or whatever, possibly transferring them off the db first if nec
[02:21:57] Swat2: maybe I need to tackle it the other way around like you suggest, extract the data to local, then scp/ftp/<insert file transfer operation here> to the remote server
[02:22:23] elomatreb: That seems like a sane choice, remote database connections in general are somewhat of codesmell usually
[02:22:24] RickHull: if you are IO bound, you'll want to be careful about saturating the db's network
[02:23:31] Swat2: RickHull: Thankfully the dbserver has two nics - one facing their internal environment and the dedicated to the remote environment
[02:24:49] Swat2: it seems to be the delay in doing constant select queries and it having to wait for the result only then to be written out to disc
[02:26:42] elomatreb: Doing it locally, paged/batched, and doing the upload to the remote separately seems like the best choice
[02:27:41] Swat2: the far end is the selector, that is, the ruby code on the remote is pulling the data.
[02:28:07] RickHull: get the bits onto the filesystem, with your actor as close to the db as possible
[02:28:32] elomatreb: Is this a recurring operation, or just a one-off script? Because this seems like a complicated and fragile design, if it's recurring
[02:31:39] Swat2: elomatreb: the DB is the "source of truth" and the code overwrites the local copies if changed/new
[02:32:12] elomatreb: This seems to get into database replication territory, which is a really old and difficult problem
[02:33:22] apeiros: swat2: in case nobody mentioned it so far - don't forget to segment the ID to build a directory tree
[02:33:41] apeiros: couple of file systems deal badly with more than a couple of thousand files in one level
[10:38:33] php_newbile: Hello I'm trying to figure out what happens when a global variable it's defined inside a function, until I call the function the variable it's empty but referencing it does not give error, after function call it's defined but also assigned within a value, am I right? "But why the defined? ()"function gives empty string before function call and "global-variable" after?
[12:36:19] oddDball: Is it OT to ask about a specific Ruby gem? I'm hoping to get some tips for speeding up prawn. 17 seconds for 20 pages (no PNGs, headers, footers, etc.) seems quite slow.
[18:01:29] nikhgupta: hey. Any library that allows end-users to add dynamic conditionals in a form? e.g. `a` > 2 AND `b` < 4 (user is able to provide a, b, 2, 4, AND via form)?
[18:02:32] ruby[bot]: nikhgupta: Please join #RubyOnRails for Rails questions. You need to be identified with NickServ, see /msg NickServ HELP
[19:48:11] lamduh: Can someone explain to me how sidekiq provides a performance benefit? https://github.com/mperham/sidekiq/wiki/FAQ
[19:48:33] lamduh: The part I am struggling with is that if the background jobs are being run on the same machine how does that lessen the performance load on the box?
[19:49:05] lamduh: Or is the primary benefit just to allow the method to be nonblocking and you don't actually net a benefit.
[19:49:47] lamduh: mikecmpbll, yeah I was about to follow up with that. Cool, so understanding it correctly I would rather use bunny or some other message broker that offloads the processing to another server if I wanted to get a performance gain?
[19:50:25] mikecmpbll: no, you're suggesting that sidekiq workers have to run on the "same machine" (as something, the web app? idk)
[19:51:10] lamduh: so you do get a performance gain on the main web app because you send the jobs to another machine with a web request
[19:51:26] lamduh: the only difference between bunny and sidekiq then would be you dont have the overhead of a message queue
[19:52:25] RickHull: lamduh: sidekiq is for when you have "background jobs" -- it's not a performance enhancement if your problem doesn't look like this
[19:52:59] RickHull: the web visitor doesn't want to wait for all the email sends to be acknowledged
[19:53:17] lamduh: right...I get that just trying to understand the difference between a message queueing solution with rabbitmq or the redis based solution with sidekiq
[19:54:20] RickHull: you can't compare it to a general "queue solution with rabbitmq" unless you define what that is
[19:54:49] lamduh: okay there is no way sending messages to redis to be consumed by worker processes on a different server is an entierly different thing than sending messages to rabbitmq to be consumed by workers on a different server
[19:56:35] dminuoso: if you dont have job semantics ("run this task tomorrow"), you probably dont need sidekiq
[19:57:25] lamduh: ah okay so that is a bit different...setting a time to run it than just consuming things isntantly off a queue...if you had rabbitmq you would proably consume the message and schedule the job separately
[19:58:04] lamduh: I was just talking to a friend that works at a big ruby startup yesterday and he brought up that they used sidekiq instead of bunny, but couldn't explain to me why
[19:58:25] dminuoso: but whether it actually does so depends on other tasks that are in the queue and availability of workers
[20:00:41] lamduh: yeah async communication through message passing...but I think you are saying sidekiq is a job queue and rabbitmq is just a general broker for async communication that could be used to build a job scheduler.
[20:15:11] RickHull: seems pretty odd to me. I get e.g. "** Invoke /tmp/Rakefile (first_time, not_needed)" with --trace
[20:16:24] RickHull: you get the expected error if the file doesn't exist: "Don't know how to build task '/tmp/Rakefile' (see --tasks)" (before I touch'd it)
[20:19:56] RickHull: is there any doc or specification for what [targets] are supposed to be? https://ruby.github.io/rake/doc/command_line_usage_rdoc.html
[20:20:50] RickHull: but I don't have any FileTasks in my Rakefile -- seems like a bug or misfeature
[20:34:24] dminuoso: ACTION is wondering whether RickHull is talking with someone on his ignore list
[20:36:24] RickHull: can anyone explain the rationale for: rake /path/to/any/existing/file_or_dir # why is this a NOOP?
[20:39:30] dminuoso: RickHull: https://github.com/ruby/rake/commit/2de58bc9e1b1ab89564f160c58f768cefa736e70
[20:40:40] dminuoso: RickHull: This appears to be quite old. My best guess is, that nobody noticed or ever cared enough to change it.
[20:43:15] dminuoso: RickHull: Im looking at how file tasks work. I'd almost say that its obvious that it should be a noop
[20:43:34] dminuoso: RickHull: as far as I can tell a file task would be skipped if the named file already exists.
[20:44:30] RickHull: I don't understand the rationale for #synthesize_file_task and I doubt the author considered the overall behavior
[20:47:57] RickHull: if a rakefile has no file tasks, why is the rake executable creating NOOP file tasks? for what purpose?
[22:15:27] elomatreb: IIRC bubblebabble is an encoding format for binary data (like base64), but designed to be human readable (as in, read out loud)
[22:16:16] ruby[bot]: elomatreb: # => undefined method `bubblebabble' for Digest::SHA256:Class (NoMethodError) ...check link for more (https://eval.in/893472)
[22:16:28] ruby[bot]: elomatreb: # => "xopoh-fedac-fenyh-nehon-mopel-nivor-lumiz-rypon-gyfot-cosyz-rimez-lolyv-pekyz-rosud-ricob-surac-tox ...check link for more (https://eval.in/893473)
[22:17:28] RickHull: it's interesting that Digest::MD5 only "shows up" in that file. I doubt that's actually the case
[22:21:25] RickHull: anyone got any good ideas for this? https://github.com/ruby/rake/issues/214#issuecomment-342010547
[22:47:11] RickHull: https://www.krautcomputing.com/blog/2015/09/21/rubys-built-in-databases-meet-pstore-and-yamlstore/
[22:47:31] RickHull: I can't remember if I looked at these before making https://github.com/rickhull/dotcfg