regedit

Activity Graph

Page 1 of 2 | Next »

2019-05-22

[10:33:28] regedit: Quit: Connection closed for inactivity

2019-05-17

[02:26:49] regedit: Quit: Connection closed for inactivity
[13:11:15] regedit: has joined #RubyOnRails
[13:11:30] regedit: has joined #ruby
[15:50:56] regedit: Quit: Connection closed for inactivity

2019-05-16

[02:57:35] regedit: has joined #RubyOnRails
[02:57:49] regedit: has joined #ruby
[07:07:05] regedit: Quit: Connection closed for inactivity
[16:17:02] regedit: has joined #RubyOnRails
[16:17:17] regedit: has joined #ruby

2019-05-14

[02:59:53] regedit: has joined #RubyOnRails
[03:00:08] regedit: has joined #ruby
[10:49:33] regedit: Quit: Connection closed for inactivity

2019-04-17

[15:08:21] regedit: has joined #RubyOnRails
[15:08:36] regedit: has joined #ruby
[23:27:45] regedit: Quit: Connection closed for inactivity

2019-04-12

[13:09:51] regedit: has joined #RubyOnRails
[13:09:57] regedit: has joined #ruby
[15:48:44] regedit: Quit: Connection closed for inactivity
[18:00:27] regedit: has joined #RubyOnRails
[18:00:41] regedit: has joined #ruby
[22:00:08] regedit: Quit: Connection closed for inactivity

2019-04-08

[05:50:04] regedit: has joined #RubyOnRails
[05:50:17] regedit: has joined #ruby
[10:49:15] regedit: Quit: Connection closed for inactivity
[13:11:08] regedit: has joined #RubyOnRails
[13:11:22] regedit: has joined #ruby
[17:30:26] regedit: Quit: Connection closed for inactivity

2019-03-13

[13:58:22] regedit: has joined #RubyOnRails
[13:58:35] regedit: has joined #ruby
[22:58:05] regedit: Quit: Connection closed for inactivity

2019-02-07

[15:31:41] regedit: has joined #RubyOnRails
[15:31:47] regedit: has joined #ruby
[19:00:59] regedit: Quit: Connection closed for inactivity

2019-01-09

[00:13:54] regedit: Quit: Connection closed for inactivity

2019-01-08

[14:44:30] regedit: has joined #RubyOnRails
[14:44:34] regedit: has joined #ruby

2018-12-20

[14:14:42] regedit: has joined #RubyOnRails
[14:14:49] regedit: has joined #ruby
[23:34:20] regedit: Quit: Connection closed for inactivity

2018-11-01

[03:14:59] regedit: has joined #RubyOnRails
[03:15:06] regedit: has joined #ruby
[08:34:35] regedit: Quit: Connection closed for inactivity

2018-09-17

[02:10:59] regedit: Quit: Connection closed for inactivity
[02:17:13] regedit: has joined #RubyOnRails
[02:17:21] regedit: has joined #ruby
[08:51:01] regedit: Quit: Connection closed for inactivity

2018-09-16

[16:41:17] regedit: has joined #RubyOnRails
[16:41:26] regedit: has joined #ruby

2018-09-11

[18:29:33] regedit: Quit: Connection closed for inactivity

2018-09-05

[14:38:31] regedit: has joined #RubyOnRails
[14:40:33] regedit: has joined #ruby
[14:41:38] regedit: HTTPrb question: how can i debug/output the raw HTTP request that gets sent?
[14:50:36] regedit: hm i see....
[14:52:04] regedit: interesting ok i'll keep those in mind thanks

2018-07-31

[07:06:00] regedit: Quit: Connection closed for inactivity

2018-07-30

[15:46:31] regedit: has joined #RubyOnRails
[15:46:41] regedit: has joined #ruby
[15:49:52] regedit: i have a HUGE string of text i'm trying to process/work with, it's giving me some problems because well, it's huge
[15:50:19] regedit: getting some memory limitations complaints (heroku ruby/rails)
[15:51:56] regedit: it's basically like 9M of phone numbers i'm trying to compare with other existing phone numbers in my database
[15:53:00] regedit: but first i fix up the huge string a bit... split it, remove some characters from it (area codes & whitespace etc), all in all it looks like the runtime piles up a lot of memory very quickly. even trying to call GC.start a few times but still memory complaints
[15:53:35] regedit: ineb: hmmm.... maybe
[15:54:08] regedit: if it helps clarify the requirements here: this is a Do Not Call list from the federal gov
[15:54:14] regedit: telemerketing.ftc.gov kinda stuff
[15:55:04] regedit: ye eventually i try making a Hash of phone-number => nil, so that i can use it as a lookup
[15:56:04] regedit: i was wondering if i need to somehow make some sort of clever compact lookup structure out of it (btree or something?) so it uses as little ram as possible
[15:56:58] regedit: but it seems all the gsub and split etc steps before might also be "conflagrating" the ram copies of the data...
[15:57:11] regedit: mnemon: possibly....?
[15:57:17] regedit: lemme show some example snippets:
[16:00:03] regedit: ok here's the entire Sidekiq Worker actually: https://kopy.io/znxKa
[16:01:02] regedit: some line numbers of possible interest: :20 fetch_file.file.read might load the entire zip compressed file into memory
[16:01:46] regedit: :30 dnc_raw = entry.get_input_stream.read copies the entire extracted string into ram?
[16:02:33] regedit: :33 trying to regex through the entire string to make sure it conforms to expected format - copies it again?
[16:02:57] regedit: :35 gsub twice
[16:07:49] regedit: im actually finding some instances of these extracted data files to contain 16M+ of data
[16:08:37] regedit: so how can i sanely process this much data within limited resources, effectively cleaning up after everything so RAM doesn't keep leaking?
[16:12:39] regedit: ineb: ah ye i was having problems giving Zip::File.open() something IO-ish, nothing seemed to work except open_buffer(fetch_file.file.read) which i think might indeed be the full string...
[16:18:31] regedit: Zip::File.open(fetch_file.file)
[16:18:49] regedit: it's expecting filename string
[16:20:12] regedit: TypeError: no implicit conversion of CarrierWave::Storage::Fog::File into String
[16:23:03] regedit: Zip::File.open_buffer(fetch_file.file)
[16:23:10] regedit: RuntimeError: Zip::File.open_buffer expects a String or IO-like argument (responds to tell, seek, read, close). Found: CarrierWave::Storage::Fog::File
[16:29:20] regedit: in this case it's the path where it's located within the storage back-end, in this case AWS S3 via fog-aws...
[16:29:45] regedit: ineb: won't StringIO.new(fetch_file.read) also cause another round of copying the string into memory?..
[16:34:46] regedit: ineb: ok so this pertains to reading & extracting the zip file, any idea if there's more to solve in this huge string processing code?..
[16:36:12] regedit: let's say we didnt even have to extract a zip file, but rather the data fetched from S3 was directly the uncompressed 16M file
[16:43:37] regedit: ineb: it causes memory complaints which in some cases triggers a killing of the process. It seems to happen "over time" after executing this process several times (not the first times it executes. memory leak-ish...)
[19:17:40] regedit: how are ruby Hash lookups implemented? is it anything special to make look ups very fast?
[19:18:29] regedit: i'm trying to find a cood collection/container to use in order to lookup simply whether (boolean true/false) a value exists in the collection or not, with super fastest lookup time possible
[19:53:30] regedit: i've been doing Hash with key => nil, because all i care for are the existence of the keys, dont care for the values
[19:53:51] regedit: so i was wondering if there's an even more optimized way of only using the keys part of it without values
[19:57:45] regedit: also, since i'm adding a huge amount of elements, which if i understand correctly the underlying mechanisms of Hash how it increases in number of underlying "bins", i wish there was a way for me to specify number of bins to begin with, so it doesnt have to keep re-dividing and re-creating Hash bins...
[19:58:39] regedit: which if i understand correctly could exponentially affect performance
[19:58:43] regedit: https://www.engineyard.com/blog/hash-lookup-in-ruby-why-is-it-so-fast
[19:59:00] regedit: "At some point you pay an exponentially increased time penalty while Ruby resizes the bin pool"

2018-07-27

[19:01:14] regedit: has joined #RubyOnRails
[19:01:24] regedit: has joined #ruby
[19:03:17] regedit: anyone use carrierwave + aws/s3? why is calling uploader.remove! not actually deleting the file from s3?
[19:03:35] regedit: anyone use carrierwave + aws/s3? why is calling uploader.remove! not actually deleting the file from s3?
[21:50:25] regedit: Quit: Connection closed for inactivity

2018-07-12

[00:21:26] regedit: Quit: Connection closed for inactivity

2018-07-11

[20:51:56] regedit: has joined #RubyOnRails
[20:52:05] regedit: has joined #ruby

2018-07-04

[18:29:03] regedit: has joined #RubyOnRails
[18:29:12] regedit: has joined #ruby
[18:31:43] regedit: (how) can i reference the request parameters within routes.rb? ex. if params['foo'] == 'bar'; match.... end
[18:32:15] regedit: NameError: undefined local variable or method `params' for #<ActionDispatch::Routing::Mapper:0x00000006112780>
[18:32:24] regedit: is what i currently get
[21:15:19] regedit: Quit: Connection closed for inactivity

2018-06-29

[05:15:38] regedit: has joined #RubyOnRails
[05:15:48] regedit: has joined #ruby
[10:24:41] regedit: Quit: Connection closed for inactivity

2018-06-23

[01:55:08] regedit: Quit: Connection closed for inactivity

2018-06-22

[23:38:57] regedit: has joined #RubyOnRails
[23:39:05] regedit: has joined #ruby

2018-06-14

[09:52:43] regedit: Quit: Connection closed for inactivity

2018-06-13

[01:05:01] regedit: has joined #RubyOnRails
[01:05:09] regedit: has joined #ruby
[03:54:10] regedit: Quit: Connection closed for inactivity
[20:33:08] regedit: has joined #RubyOnRails
[20:33:16] regedit: has joined #ruby

2018-06-12

[02:13:00] regedit: has joined #RubyOnRails
[02:13:08] regedit: has joined #ruby
[08:52:34] regedit: Quit: Connection closed for inactivity

2018-06-08

[08:29:05] regedit: Quit: Connection closed for inactivity

2018-06-07

[01:03:36] regedit: anyone? how can i do this the Sidekiq way https://github.com/httprb/http/wiki/Thread-Safety
[01:04:07] regedit: trying to persist connection (keep-alive) across many executions of a sidekiq worker
[01:05:03] regedit: ex. imagine queueing up a bunch of Sidekiq tasks each of which performs an API call to the same host
[07:41:34] regedit: Quit: Connection closed for inactivity
[17:36:12] regedit: has joined #RubyOnRails
[17:36:20] regedit: has joined #ruby

2018-06-06

[04:56:09] regedit: has joined #RubyOnRails
[04:56:17] regedit: has joined #ruby
[04:57:36] regedit: i once came asking for a recommendation for an HTTP library, and was advised to use Typhoeus
[04:57:47] regedit: with its hydra parallel requests feature
[04:58:11] regedit: problem is - i now realize i want to make these HTTP requests from a Sidekiq worker....
[05:00:56] regedit: the sidekiq worker performs its task on 1 record at a time, rather than having all records to process and queue up for typhoeus http requests... so it seems i might be losing out on running many Typhoeus+hydra requests in parallel
[05:02:58] regedit: is there another way of queuing up many many requests for typhoeus in parallel from within workers that operate on only 1 record at a time?
[05:19:24] regedit: havenwood: so persistent http connection is not an applicable notion for me if my records are being processed 1 at a time in sidekiq workers?
[08:04:39] regedit: Quit: Connection closed for inactivity
[13:02:29] regedit: has joined #RubyOnRails
[13:02:37] regedit: has joined #ruby
[13:04:40] regedit: havenwood: still there? do i need to do anything to expressly use such a connection pool? or it happens behind the scenes
[13:06:16] regedit: ah nvm i think my answer should be here https://github.com/mperham/sidekiq/wiki/Advanced-Options#connection-pooling
[13:08:24] regedit: havenwood: is your recommendation to use connection_pool with HTTP.rb or Typhoeus?
[13:16:22] regedit: Silex: any idea how do use HTTP.rb in conjunction with (sidekiq with) connection_pool ?
[13:19:11] regedit: oh lol your above URL discusses exactly that :D thanks
[13:22:24] regedit: the last mystery to solve would be how to do this https://github.com/httprb/http/wiki/Thread-Safety in a sidekiq worker so that the http connection is persisted across hundreds of distinct sidekiq worker executions...
[13:22:54] regedit: and then somehow close the connection pool at the right time after the batch of said sidekiq jobs have completed
[22:55:23] regedit: havenwood: hey
[22:58:14] regedit: oh cool, what conference?
[22:59:04] regedit: is anyone successfully using Sidekiq + connection_pool + HTTP.rb to make a bunch of keep-alive requests across many distinct executions of a Sidekiq Worker?

2018-06-01

[06:51:01] regedit: Quit: Connection closed for inactivity

2018-05-31

[18:01:30] regedit: has joined #RubyOnRails
[18:01:38] regedit: has joined #ruby
[18:04:08] regedit: where do i put a misc/util method that i need to call from various different places in the codebase?
[18:04:48] regedit: in this case: several different Sidekiq workers need to clean up & fix some CSV files before being able to do with them whatever each worker processes
[18:12:16] regedit: these workers do some long running spreadsheet data processing
[18:12:53] regedit: at the beginning of these workers they need to open spreadsheet files of course. Using the 'roo' gem, sometimes it's a xls file, xlsx, csv
[18:13:43] regedit: CSVs are a pain because they often come malformed, including various bad characters, and other problems needing cleaning - otherwise they dont open with roo...
[18:15:43] regedit: right now i have some 10 lines of CSV cleaning code that is repeated identically in 3 distinct worker classes...
[18:17:11] regedit: since they're not really serving the MVC side of the rails app, i guess i should put such generic/utility methods under lib/ ?

2018-05-24

[03:49:43] regedit: has joined #RubyOnRails
[03:49:51] regedit: has joined #ruby
[04:07:51] regedit: How can i query an Author's related Posts where the Posts' related Comments exist where some criteria applies?
[04:17:21] regedit: kind of like in SQL: select * from authors a join posts p on a.id = p.author_id where exists (select * from comments where post_id = p.id and not trashed)
[04:17:46] regedit: kind of like in SQL: select * from authors a join posts p on a.id = p.author_id where exists (select * from comments where post_id = p.id and not trashed)
[04:17:58] regedit: whoops - my bad
[04:18:27] regedit: how can i do such an ActiveRecord query the RoR way?
[09:19:14] regedit: Quit: Connection closed for inactivity

2018-05-23

[15:50:51] regedit: Quit: Connection closed for inactivity

2018-05-22

[21:51:22] regedit: has joined #RubyOnRails
[21:51:30] regedit: has joined #ruby
[22:04:36] regedit: I have a Messages model and ContactNumber model. I'd like to add ContactNumber.opted_out_message_id column which should refer to a single Messages record
[22:04:48] regedit: The idea being i need to record whether a ContactNumber is considered "opted out", and if so which Message record is the one that "opted out" the ContactNumber
[22:05:55] regedit: i'm fumbling around not sure how to achieve this, messing with belongs_to / has_one, add_reference with custom column name, no foreign keys etc...

2018-05-18

[00:22:32] regedit: Quit: Connection closed for inactivity
[15:12:08] regedit: has joined #RubyOnRails
[15:12:16] regedit: has joined #ruby
[18:31:39] regedit: Quit: Connection closed for inactivity

2018-05-17

[16:03:32] regedit: has joined #RubyOnRails
[16:03:40] regedit: has joined #ruby

2018-05-16

[00:57:53] regedit: Quit: Connection closed for inactivity
[15:02:35] regedit: has joined #RubyOnRails
[15:02:43] regedit: has joined #ruby
[23:48:48] regedit: Quit: Connection closed for inactivity

2018-05-15

[16:58:18] regedit: has joined #RubyOnRails
[16:58:28] regedit: has joined #ruby
[17:01:27] regedit: if i'm using a gem like Unirest and i know i'm gonna be placing hundreds of HTTP requests to a given 3rd party, do i have to do anything to care about persistent connection / keep alive?
[17:01:29] regedit: Or will that be taken care of automagically
[17:09:54] regedit: wow nice list :+1:
[17:10:25] regedit: not sure if i want/need parallel... maybe
[17:10:54] regedit: i'm gonna be interacting with an API, processing hundreds of records, each one its own API call
[17:15:22] regedit: ok cool thanks havenwood
[19:27:53] regedit: Quit: Connection closed for inactivity
[20:19:15] regedit: has joined #RubyOnRails
[20:19:22] regedit: has joined #ruby
[20:23:09] regedit: i have 2 Sidekiq::Worker classes, i'd like both of them to perform some interactions with a 3rd party API, rather new to Ruby & RoR, how should i structure this? Create some sort of DoTheApiThing class which both workers will use?
[20:42:39] regedit: what about lib/ ? sorry still rather new to RoR.... any links to relevant docs?
[22:30:36] regedit: Radar: GOOD MORNING to you too! XD Thanks!
[22:31:29] regedit: Radar: someone else suggested i drop it in Models :shrug: any difference ?
[22:32:51] regedit: app/workers? isn't that where actual worker classes should go, the kinds that are supposed to be perfom()ed as scheduled jobs?
[22:33:27] regedit: ACTION pours some hawaiian magma on Radar's face
[22:34:20] regedit: ok so i make something like lib/apithing.rb containing class ApiThing ?
[22:34:52] regedit: then i require apithing at the top of app/workers/my_process_worker.rb ?
[22:35:16] regedit: awesome :+1: thanks!
[22:35:46] regedit: ye it seems rather very paranoid-ly protected & secured... almost NDA-like
[22:36:21] regedit: https://www.attomdata.com/
[22:38:37] regedit: yep... all behind paid pricey membership deals...
[22:39:08] regedit: get this - the API is restricted by IP
[22:39:32] regedit: o_O personally i've never seen/heard such a level of API paranoia before
[22:41:17] regedit: https://usercontent.irccloud-cdn.com/file/LRtI3WF8/ParanoiaAPI

2018-05-11

[03:09:33] regedit: Quit: Connection closed for inactivity
[16:37:02] regedit: has joined #RubyOnRails
[16:37:13] regedit: has joined #ruby
[16:39:00] regedit: why am i getting errors if i'm regex-ing for a hash/pound symbol at the end of a string, ex. /foo#$/
[16:39:09] regedit: SyntaxError: (irb):121: unterminated regexp meets end of file
[16:39:28] regedit: but /foo\#$/ seems to work
[16:39:36] regedit: reason for this?
[16:51:10] regedit: but this is in middle of a regex, you'd think that would be picked up by the compiler..
[16:51:20] regedit: parser/lexer/whatever
[19:26:38] regedit: Quit: Connection closed for inactivity

2018-05-10

[00:15:46] regedit: Quit: Connection closed for inactivity
[22:50:04] regedit: has joined #RubyOnRails
[22:50:14] regedit: has joined #ruby
[22:51:41] regedit: i'd like to conditionally pass one set or another set of optional parameters into a method. Ex: Thing.foo(lol: 1, rofl: 2) OR Thing.foo(haha: 10, wtf: 20)
[22:52:42] regedit: is it possible to do this without calling the method in an "if" and again in the "else" ?
[22:53:31] regedit: let's say it boils down to a boolean expression
[22:55:14] regedit: havenwood: can you explain or demonstrate what you mean in your 2nd suggestion about assigning pairs and passing as **param
[23:00:37] regedit: aha! bingo, i was hoping there was a way of passing a sort of {temporary: "object"} which will be treated as if it were (comma, separated, parameters)
[23:00:43] regedit: ACTION is rather new to ruby
[23:19:27] regedit: k thanks guys 👍

2018-05-09

[16:52:49] regedit: sylario: ehm well this is actually on heroku... so, puma or something..
[16:54:03] regedit: any clue about that?
[16:59:33] regedit: not sure what you mean keep relative path
[17:02:30] regedit: on heroku i dont believe i have access to mess with DNS settings anyway (i'm using the herokuapp.com default free domain address...)
[17:08:29] regedit: googling around a bit, so far i've come up with:
[17:08:48] regedit: in config/routes.rb
[17:08:56] regedit: match '*path', to: redirect(...???...), via: :all
[17:09:45] regedit: if this is the right way to go, then i just need to figure out how to make it redirect with all the path + query stuff, and HTTP status code 307 (or 308)
[17:29:31] regedit: in routes.rb how do i refer to the request URI path & query? i'm trying to redirect the request to a different subdomain but keeping all the other URI components (path / params / query / fragment etc)