Skip to main content
16 events
when toggle format what by license comment
Oct 2, 2022 at 18:17 comment added Karl Knechtel "How on earth does data processing have nothing to do with programming" Because the question does not tell us anything about the programming that will be done with that data - how it will be processed. So far, the only thing we know about the data is that it has previously been sent to OP via email, and that it needs to be stored in an s3 bucket. Neither of those things involves writing code.
Oct 2, 2022 at 18:16 comment added Karl Knechtel The question gives us no idea what your goals or requirements are, so there is no way we can possibly decide what format would be better or worse for OP. Multiple data formats exist for a reason; they have strengths and weaknesses according to the purpose of the data etc. We can't possibly tell OP what questions to ask; that could fill a book.
Oct 2, 2022 at 7:59 comment added Dharman Mod @MartinJames sure, if they lack details you can still vote to close
Oct 2, 2022 at 5:04 comment added Martin James ....so, many of these 'opinion' qurstions are vague and underspecified. They lead to chat conversations instead of solutions.
Oct 2, 2022 at 5:01 comment added Martin James 'Why is it better to write A instead of B?' Better for what? 'better', for me, usually means 'easier to debug', YMMV.
Oct 2, 2022 at 4:58 comment added Martin James 'What would be the most correct way to write this function?' Define 'correct'. If you have to deliver tomorrow and your first, simple solution works, though slow and inefficient, then that is the correct way.
Oct 2, 2022 at 4:54 comment added Martin James 'How can I make my algorithm faster?' Faster to execute? On what hardware? Do you have enough memory for a lookup-table? Is there enough data to thread off to a pool? Did you mean faster to develop, test and debug?
Oct 1, 2022 at 21:29 comment added EJoshuaS - Stand with Ukraine all answers (and therefore all questions) are necessarily opinion based answers That's just your opinion.
Oct 1, 2022 at 14:37 comment added Clive “As though that is what will increase quality”…I think it’s more “that is what will stem the tide”. SO is already overrun with crap, we need ways to stop it. If that means that some subjects which, in a perfect world, would otherwise have been viable for the site get sidelined, then so be it. If everyone played by the rules, I think you’d have a good point about this data processing question potentially being programming related, at least implicitly. But we don’t live in that world, SO needs a narrow scope just to survive these days IMO
Oct 1, 2022 at 13:59 comment added ticster I mean really this is where we part ways. The general attitude at SO is to narrow what's acceptable as much as possible (like considering data storage in data engineering to be non programming related) as though that is what will increase quality. And I find the correlation, at least these days, to run the other way. Most of the garbage is hyper specific, and the few quality questions are broader in scope.
Oct 1, 2022 at 12:42 comment added Dharman Mod The question isn't asking how to write software to process the files. It just asks what file format to store information in. That's it. No programming involved. They just want to store a file on a server.
Oct 1, 2022 at 12:19 comment added ticster How on earth does data processing have nothing to do with programming ?
Oct 1, 2022 at 11:56 comment added Dharman Mod @ticster That question is not only opinion-based, but also too broad and off-topic because it doesn't seem to have anything to do with programming.
Oct 1, 2022 at 9:17 comment added ticster In other words, by all means close polls and pointless "is C++ better than Python" questions. But there is a broad consensus that any question that touches on best practices should be closed, and that's a bad idea, if only because de facto most answers implicitly have a smattering of best practice recommendations in them.
Oct 1, 2022 at 8:53 comment added ticster This is the question that spurred my post (not my question, but it got closed after I gave a fairly comprehensive answer). It's a clear question about common things to look out for when receiving data dumps. It's somewhat broad for sure, but is more interesting than 80% of new questions these days. It still got closed, even though it's basically just asking "what specifications are required for this type of task".
Oct 1, 2022 at 8:17 history answered DharmanMod CC BY-SA 4.0