-16

Yesterday I posted What is the best way / algorithm to make this sequential sorting code more performant (deleted) and it was closed with needs to be more focused mark (It had a mistake and was not clear enough). I posted a new question Is it possible to improve the performance of this code but with several edits based on advises in Why was my question about improving the performance of sequential sorting code in NumPy closed as lacking focus? : shortening the code, explaining the goal again in an understandable way, more focused and removed suspicious parts relating to probable multiple answers, but it closed again. The new post was:

Title: Is it possible to improve the performance of this code

I have a 2D array mod_arr with shape (m*n) that contains integers/indices and need to be modified (values in each row will be swapped in that row) based on some comparisons (sorting schemes) on values in the 3rd dimension (r) of a 3D NumPy array with shape (m*n*r) nav_org; In the 3D array the inner 2D arrays are considered independent of each other in comparisons. Rows will be swapped in each of these inner 2D arrays based on some sequential comparisons for the columns’ values. So, I first modify the 3D array and then based on the changes that were applied on this array, I modify the 2D integer array. So, in each of the inner 2D arrays:

  1. Sorting based on the 4th column in reverse order
  2. Sorting the first 7 rows based on the 2nd column in reverse order (Such separations are for comparing independently i.e., independent from values in the rest 3 rows, where nums_div = 10 --> comparison will be among just these 7 rows)
  3. Sorting the rest 3 rows based on the 1st column
CODES

As this code will be called many times in my simulation, it will be better if could be more performant (now, for nums=1000000, params=4, nums_div=10, it takes around 0.8 S, where 11 hours will be consumed by that if we call just the body that shown on the code 50000 times by a system with corei5 CPU and memory 16 Gb). I am seeking for any other faster intelligent way if possible, without Cython, with NumPy or Numba, to get res (np.int64), where nav and comparations must be based on at least np.float64 (or more precise if it could, not the main goal; any recommendation will be appreciated in this regard).
Notes:

  • For params and nums_div, the mentioned values (3 and 7 respectively) are almost used, so consider them instead the minimum values for benchmarks.
  • The proposed code, preferably, be applicable on the both Python versions 2 and 3; Python 3 is the first priority.
  • Be careful about choosing nums and nums_div which must satisfy nums % nums_div = 0 for data creation in the prepared example.

I couldn't figure out the main issue with that to improve that or post a new question in the true manner.

Please help further to clarify the issues with my post.

30
  • 1
    Link text should be help people to understand what will they found when following the link. When the links points to SO questions IMHO the best is to keep the question title.
    – Wicket
    Commented Jan 2, 2023 at 3:51
  • 12
    When a question is closed, it's a bad practice to delete the question and post an improved version as a new question, instead, edit the closed question and wait for it be reopened. If you repeat the bad practice you might got a question post ban.
    – Wicket
    Commented Jan 2, 2023 at 3:53
  • 2
    @Rubén I didn't know that. I just did what the warning said You can edit the question or post a new one. I will consider it. I will undelete the ex SO post if needed.
    – Ali_Sh
    Commented Jan 2, 2023 at 4:02
  • 3
    Don't forget that code review exists for code that is already correct but could use ideas for improvement, including performance gains. Commented Jan 2, 2023 at 4:50
  • 2
    @PresidentJamesK.Polk Right, I've little experience with that but it must be said (as I wrote in my SO question' comments): I really follow some experts on SO who are very professional at performance (as user:12939557), and you can see some of their answers on my ex much more complicated questions, who are not active on Code Review, who I believe on them and I think their answers on my asked topics will probably be the best. I will try that, but hope SO reopen my question, because I really did/do not think that my question was so inappropriate to be closed there. tnx.
    – Ali_Sh
    Commented Jan 2, 2023 at 4:58
  • 6
    @Ali_Sh no? With a title that literally begins with the words "What is the best way"? The bullets were already in the air at that point.
    – Gimby
    Commented Jan 2, 2023 at 9:32
  • @Gimby , I've changed the title, hope it help.
    – Ali_Sh
    Commented Jan 2, 2023 at 14:57
  • It seems the issues in my post is under the magnifying glass. I am editing my post for 2 days by the feedbacks that I get on this forum and SO, but again I get negative feedbacks. I doubted that if my edited post is seen or not. And I don't know what else should I do?
    – Ali_Sh
    Commented Jan 2, 2023 at 16:31
  • 3
    Ultimately, you have stumbled upon a kind of question which does not make a great fit for the Stack Exchange network. The site itself could perhaps do a better job at communicating this. It is also true that the advice on editing or posting a new question is indeed bad and has been strongly contested. Should I be the one with the problem at hand, I would research for one feasible approach to optimizing the heavy load of the program (for example, use Numba) and eventually ask more specifically about it.
    – E_net4
    Commented Jan 2, 2023 at 16:44
  • 1
    Your closed question was deleted by you. Why? Asking a question then deleting it (instead of editing it) is toxic behavior I can’t review it and provide feedback because you deleted it. Commented Jan 3, 2023 at 0:14
  • 1
    @Ali_Sh - You deliberately deleted your question. Which is the reason your second question was closed. Commented Jan 3, 2023 at 5:24
  • 1
    "closers must be less strict and be a little flexible due to the internationality of this forum" Note that the question was not closed due to grammatical mistakes or any of the sort. Questions are not meant to be voted based directly on the author's reading and writing skills either. We also are not fond of deliberately making exceptions to curation. They just make things even harder to curate, precisely because people point at such questions and say "then why isn't that one closed??".
    – E_net4
    Commented Jan 3, 2023 at 9:54
  • 2
    Calling this place a forum is misguided too. There are plenty of venues which are open to broader discussion about a problem, but this one isn't.
    – E_net4
    Commented Jan 3, 2023 at 9:55
  • 2
    You can edit the question or post a new one. is, and pretty much always has been, horrible advice that gets new users banned. Can we PLEASE change that message? Pretty please with wombats on top? Commented Jan 3, 2023 at 23:58
  • 2
    @user4581301 asking nicely has never done anything. Try an angry tweet.
    – Gimby
    Commented Jan 4, 2023 at 9:38

1 Answer 1

17

Because your question basically boils down to this

As I guess the chosen algorithm will affect the performance, significantly, for this code, I am seeking for the fastest intelligent algorithm

This needs a lot more focus to be answerable. On top of that, we generally don't allow open-ended discussion of what algorithms to use. Especially

The performance is the main objective.

We won't know how this works with your setup. Performance is quite subjective, unfortunately. And answers are likely to be of the opinion variety.

If you can come back with a section of code that appears to be under-performing (with some stats and settings to try and replicate the problem), it might be answerable.

9
  • I added the updated question into this issue, the link was attached incorrectly, I added it correctly now. please see that. In the new question, I have removed it.
    – Ali_Sh
    Commented Jan 2, 2023 at 3:34
  • stackoverflow.com/questions/74977526/…
    – Ali_Sh
    Commented Jan 2, 2023 at 3:34
  • I have written The performance is the main objective. to say that more precision is not the first priority. the first is performance with considering the specified limits i.e. ranges and float type and … .
    – Ali_Sh
    Commented Jan 2, 2023 at 3:42
  • 1
    before anything else, I appreciate for you participation for help. Another matter is that I have mentioned the expected methods (with NumPy or Numba) for use in my new post. And also mentioned that this code will be called many times, which will be take more than12 hours based on the problem sizes (I did not mentioned how long it could take).
    – Ali_Sh
    Commented Jan 2, 2023 at 3:58
  • "I have written The performance is the main objective. to say that more precision is not the first priority." That needs to be quantified. Otherwise I can write code that returns an array of the appropriate shape filled with zeros, and consider the task complete. Commented Jan 4, 2023 at 0:42
  • @KarlKnechtel In the post I have mentioned float64 for dtype is enough, by more precise I mean like float128 but if it doesn’t affect, adversely, the performance a lot. So, I said/meant performance with float64 is the main concern and competition will be based on that. I wrote a code that needs to be improved, so just using np.zeros instead np.empty or … in the code to improve it even a little, will be accepted if not any more performant answer be written (I’ve removed The performance is the main objective. based on this answer).
    – Ali_Sh
    Commented Jan 4, 2023 at 17:36
  • I did not catch why it was open-ended discussion, I’ve wrote that after explanations about float64 or float128, perhaps badly written? In the other hand, I didn't have any idea how much it could speed up. For example, if the code is consuming 12 hours now (11 hours is benched for the example), speeding up the code as much as possible is expected, but if I write expected time e.g., below 1 hours in the post (which is good enough), answerers will not put their answer if they think cannot reach it (although their code will be much performant but not gaining that, like 2 hours) (cont,)
    – Ali_Sh
    Commented Jan 4, 2023 at 17:38
  • even can be better than mine and acceptable (and I need that). I think an answer there, even with np.zeros instead np.empty if get better result, led to start the competition and encourage others to participate to beat that answer, if the goal of people here be to help and not to punish askers for any deficiency in his/her question, as I believe it is. if not, I must accept that. Please let me know how to request for such performance questions to get the best possible answer to avoid others blaming my question.
    – Ali_Sh
    Commented Jan 4, 2023 at 17:38
  • As my question needs Numba, Code_Review is not a good choice, because it has, so far, less than 30 answers with this tag, many not good enough IMO, and many Numba related professionals are not active there (just active on SO), so SO is much more preferable than CR. Another thing is that my question is just a small part of my main code, short enough (section of code that appears to be under-performing), IMO it fits the SO. I’ve added my system setup to show what is intended, too. My code was around np.argsort so it was not a general question, and it needs to well-handle that. Closed again.
    – Ali_Sh
    Commented Jan 4, 2023 at 17:41

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .