Page 1 of 5 | Next »
[20:48:04] gheegh: Hey all.. question. I have a problem where my sidekiq processes just seem to grow and grow and grow.. I've spent a week or so profiling, and i think the problem boils down to that we do retrievals, and when we get several big ones at once, the ruby process consumes more memory. .and never releases it. So, i'm wondering, has anyone ever FORKED in the middle of a sidekiq process (to wrap the retriever part), so i can then
[14:19:56] gheegh: Hey all.. I'm wondering if anyone knows of any guide on best practices for jobs that "retrieve data"? I have a system that's doing hundreds of thousands of jobs.. occasionally.. my Sidekiq job gets locked up.. and then re-queues itself, then it just seems to cascade.. what i need is a clean Timeout that will, after a fixed period of time, re-queue to a problem queue. Over the course of a day, it will lock up all my worker
[14:33:42] gheegh: yeah, but Sidekiq writes all over their code base not to use Timeout because it's not thread safe.
[14:34:09] gheegh: to timeout the underlying process, retrieve, etc. which i've done.. but there are still some case where it locks the thread..
[00:18:00] gheegh: hey all, anyone have a suggestion as to how to "memoize" a dynamic singleton class variable?
[00:19:55] gheegh: I've got a folder that has a bunch of files in it that i'm converting into class methods on a class. I just realized that one of hte problems is that its reloading them from disk every time it's called, versus just 1 time. I need to "memoize" them.. but i'm not sure exactly how to do that. Here's a gist.. imagine the files as something that i'm turning into Arrays..
[00:45:51] gheegh: so, this works.. its just that it seems to reload the method everytime it's called
[18:57:00] gheegh: Anyone work with active_admin? I've got a problem with a custom form. It's not rendering well, and fields are not showing. Rails 5.2 latest everything. https://gist.github.com/wflanagan/5f73eab4dfdd93aadf74d98fa4c62ba5
[19:02:06] gheegh: maybe I'll ask here, as it's maybe busier.. i have an ActiveAdmin issue. My custom form is totally not working.
[19:02:31] gheegh: It's not rendering well, and fields are not showing. Rails 5.2 latest everything. https://gist.github.com/wflanagan/5f73eab4dfdd93aadf74d98fa4c62ba5 gist shows a link to a pic, and the rendered HTML (and source of form)
[13:06:16] gheegh: hey all.. i need some thoughts. I'm trying to dynamically define a class as a child of OpenStruct, if it's not defined. If defined, I want to just use it. Basically, i'm trying to wrap OpenSturct for responses to an API call. I'm getting a nameerror when trying to use my defined class. thoughts on how to do this?
[13:07:46] gheegh: sorry, to follow the guide, here's the code snippet defined at dpaste.de https://dpaste.de/OEJz#L
[21:03:29] gheegh: Hey all, i'm having a problem with VCR in some of my tests.. i am doing the query in the "let", but it seems to retrieve each time no matter what.. any thoughts on how to fix this in RSpec?
[21:17:23] gheegh: hey all.. anyone using VCR? I'm having some problem figuring out what to do to get my "let" in RSPEC not re-retrieve the page..
[21:18:41] gheegh: https://gist.github.com/wflanagan/59f0434d11079ace97ed4679d503d31d this is an example.. not sure what i'm doing wrong.. the URL is identical.
[21:47:44] gheegh: yeah, but i would have thought vcr would have matched it.. and returned the value
[19:49:31] gheegh: hey all.. question. I'm using Moneta to bulid a simple gem that looks up/loads a file that's about 800MB uncompressed. I am storing it on Github, so I need to break it into smaller file sizes. I had loaded it into several small SQL files, but that's not supported on Heroku. The File store breaks each key into its own file, adding about 400,000 files to the github repo (not a great idea for performance). I need something smaller,
[16:26:43] gheegh: Hey all, anyone have a good, reliable config for using StatsD with ActiveJob? It seems my stats are not getting sent reliably.
[15:15:53] gheegh: theoretical question.. I have a list of about 200K records that I need to access.. it takes about 500 MB to store the entire file (in JSON format). I'm considering breaking it into a number of smaller files, zipped, (like 65k files).. and then retrieve each file from the disk and get teh record in question.. is this a terrible idea.. if it's in a gem?
[15:17:16] gheegh: the goal is to reduce the IO of storing all this in a file.. if i have to laod the entire file each time, tha'ts a lot of IO
[15:20:12] gheegh: i'm working on a research project that is realted to Twitter usernames.. i'm retrieiving mentions in a stream... and processing the user account.. the "cache" is the verified users.. about 200K that are likley to be hit.. i'd like to hit this cache isntead of retrieving their account from teh API
[15:22:36] gheegh: part of the thing I want to do is compress it.. so it doesn't take up so much space on disk
[15:23:51] gheegh: as it can gate performance loading a 500MB file each time I look up things from disk..
[15:38:39] gheegh: essentailly, I'll be checking the database into the repo.. so the problem with DBM is it's machine dependent.
[15:57:49] gheegh: I would love to be thinking in those terms.. too much going on right now.. i'm elbow deep in todos.. :-)