Skip to main content
added 48 characters in body
Source Link
Makoto
  • 106.2k
  • 120
  • 864
  • 1.3k

This is why I have a personal problem with MCVE. It fosters a scenario in which the problem you wish to solve becomes unaskable because you cannot fulfill our criteria when it would otherwise be perfectly on-topic.

Let's really get to the spirit of what it is you're asking instead.

I have a large file I need to work with, but I can't fit the entirety of it into memory. I'm working in R. How would I approach this?

Your initial thought is right; breaking it into pieces that can fit into memory is probably the right approach. You're merely now looking for the right way to go about doing that.

If I were you, I'd phrase the question almost how you already have it here.

I have a very large file in a format that I can not read or work with (Stata .dta). I have conversion software to get it into R (haven), but the file is too big to put in my RAM (about 30 GB). Is there a specific approach I can take to break my data file up so that it can fit in memory through R?

If someone wanted to reproduce this exact scenario on your behalf - complete with a large data set - there's a very good chance that they haven't had to solve this problem before, and that their answer would be a literal guess. My impression is that the experts before you who have actually addressed this problem would be able to understand it and answer it for you.

If it gets closed before then...direct them to this Meta question. It'd be useful to understand why they felt like this question needed to have more in order for them to answer.

This is why I have a personal problem with MCVE. It fosters a scenario in which the problem you wish to solve becomes unaskable because you cannot fulfill our criteria.

Let's really get to the spirit of what it is you're asking instead.

I have a large file I need to work with, but I can't fit the entirety of it into memory. I'm working in R. How would I approach this?

Your initial thought is right; breaking it into pieces that can fit into memory is probably the right approach. You're merely now looking for the right way to go about doing that.

If I were you, I'd phrase the question almost how you already have it here.

I have a very large file in a format that I can not read or work with (Stata .dta). I have conversion software to get it into R (haven), but the file is too big to put in my RAM (about 30 GB). Is there a specific approach I can take to break my data file up so that it can fit in memory through R?

If someone wanted to reproduce this exact scenario on your behalf - complete with a large data set - there's a very good chance that they haven't had to solve this problem before, and that their answer would be a literal guess. My impression is that the experts before you who have actually addressed this problem would be able to understand it and answer it for you.

If it gets closed before then...direct them to this Meta question. It'd be useful to understand why they felt like this question needed to have more in order for them to answer.

This is why I have a personal problem with MCVE. It fosters a scenario in which the problem you wish to solve becomes unaskable because you cannot fulfill our criteria when it would otherwise be perfectly on-topic.

Let's really get to the spirit of what it is you're asking instead.

I have a large file I need to work with, but I can't fit the entirety of it into memory. I'm working in R. How would I approach this?

Your initial thought is right; breaking it into pieces that can fit into memory is probably the right approach. You're merely now looking for the right way to go about doing that.

If I were you, I'd phrase the question almost how you already have it here.

I have a very large file in a format that I can not read or work with (Stata .dta). I have conversion software to get it into R (haven), but the file is too big to put in my RAM (about 30 GB). Is there a specific approach I can take to break my data file up so that it can fit in memory through R?

If someone wanted to reproduce this exact scenario on your behalf - complete with a large data set - there's a very good chance that they haven't had to solve this problem before, and that their answer would be a literal guess. My impression is that the experts before you who have actually addressed this problem would be able to understand it and answer it for you.

If it gets closed before then...direct them to this Meta question. It'd be useful to understand why they felt like this question needed to have more in order for them to answer.

added 4 characters in body
Source Link
Makoto
  • 106.2k
  • 120
  • 864
  • 1.3k

This is why I have a personal problem with MCVE. It fosters a scenario in which the problem you wish to solve becomes unaskable because you cannot fulfill our criteria.

Let's really get to the spirit of what it is you're asking instead.

I have a large file I need to work with, but I can't fit the entirety of it into memory. I'm working in R. How would I approach this?

Your initial thought is right; breaking it into pieces that can fit into memory is probably the right approach. You're merely now looking for the right way to go about doing that.

If I were you, I'd phrase the filequestion almost how you already have it here.

I have a very large file in a format that I can not read or work with (Stata .dta). I have conversion software to get it into R (haven), but the file is too big to put in my RAM (about 30 GB). Is there a specific approach I can take to break my data file up so that it can fit in memory through R?

If someone wanted to reproduce this exact scenario on your behalf - complete with a large data set - there's a very good chance that they haven't had to solve this problem before, and that their answer would be a literal guess. My impression is that the experts before you who have actually addressed this problem would be able to understand it and answer it for you.

If it gets closed before then...direct them to this Meta question. It'd be useful to understand why they felt like this question needed to have more in order for them to answer.

This is why I have a personal problem with MCVE. It fosters a scenario in which the problem you wish to solve becomes unaskable because you cannot fulfill our criteria.

Let's really get to the spirit of what it is you're asking instead.

I have a large file I need to work with, but I can't fit the entirety of it into memory. I'm working in R. How would I approach this?

Your initial thought is right; breaking it into pieces that can fit into memory is probably the right approach. You're merely now looking for the right way to go about doing that.

If I were you, I'd phrase the file almost how you already have it here.

I have a very large file in a format that I can not read or work with (Stata .dta). I have conversion software to get it into R (haven), but the file is too big to put in my RAM (about 30 GB). Is there a specific approach I can take to break my data file up so that it can fit in memory through R?

If someone wanted to reproduce this exact scenario on your behalf - complete with a large data set - there's a very good chance that they haven't had to solve this problem before, and that their answer would be a literal guess. My impression is that the experts before you who have actually addressed this problem would be able to understand it and answer it for you.

If it gets closed before then...direct them to this Meta question. It'd be useful to understand why they felt like this question needed to have more in order for them to answer.

This is why I have a personal problem with MCVE. It fosters a scenario in which the problem you wish to solve becomes unaskable because you cannot fulfill our criteria.

Let's really get to the spirit of what it is you're asking instead.

I have a large file I need to work with, but I can't fit the entirety of it into memory. I'm working in R. How would I approach this?

Your initial thought is right; breaking it into pieces that can fit into memory is probably the right approach. You're merely now looking for the right way to go about doing that.

If I were you, I'd phrase the question almost how you already have it here.

I have a very large file in a format that I can not read or work with (Stata .dta). I have conversion software to get it into R (haven), but the file is too big to put in my RAM (about 30 GB). Is there a specific approach I can take to break my data file up so that it can fit in memory through R?

If someone wanted to reproduce this exact scenario on your behalf - complete with a large data set - there's a very good chance that they haven't had to solve this problem before, and that their answer would be a literal guess. My impression is that the experts before you who have actually addressed this problem would be able to understand it and answer it for you.

If it gets closed before then...direct them to this Meta question. It'd be useful to understand why they felt like this question needed to have more in order for them to answer.

Source Link
Makoto
  • 106.2k
  • 120
  • 864
  • 1.3k

This is why I have a personal problem with MCVE. It fosters a scenario in which the problem you wish to solve becomes unaskable because you cannot fulfill our criteria.

Let's really get to the spirit of what it is you're asking instead.

I have a large file I need to work with, but I can't fit the entirety of it into memory. I'm working in R. How would I approach this?

Your initial thought is right; breaking it into pieces that can fit into memory is probably the right approach. You're merely now looking for the right way to go about doing that.

If I were you, I'd phrase the file almost how you already have it here.

I have a very large file in a format that I can not read or work with (Stata .dta). I have conversion software to get it into R (haven), but the file is too big to put in my RAM (about 30 GB). Is there a specific approach I can take to break my data file up so that it can fit in memory through R?

If someone wanted to reproduce this exact scenario on your behalf - complete with a large data set - there's a very good chance that they haven't had to solve this problem before, and that their answer would be a literal guess. My impression is that the experts before you who have actually addressed this problem would be able to understand it and answer it for you.

If it gets closed before then...direct them to this Meta question. It'd be useful to understand why they felt like this question needed to have more in order for them to answer.