NumPy: Erroneous result when modifying an array using itself
import numpy as np
bc = np.arange(10) # [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
# regular way using NumPy function
np.diff(bc) # array([1, 1, 1, 1, 1, 1, 1, 1, 1])
# something similar with array subtraction:
bc[1:] - bc[:-1] # array([1, 1, 1, 1, 1, 1, 1, 1, 1])
# but this does the wrong thing:
bc[1:] -= bc[:-1] # array([0, 1, 1, 2, 2, 3, 3, 4, 4, 5])
As a C and C ++ programmer, I can understand why this is happening (it's memcpy () instead of memmove () again), but it looks like Python and NumPy end users may not expect this. And I haven't found any documentation that says it won't work.
The question is, is there a bug in NumPy here (maybe not), or is there some NumPy documentation explaining what rules are in such situations, or is it missing from the documentation?
Second, I would like to find a safe working solution that is quasi-optimal in space and time. That is, it shouldn't allocate memory beyond a constant value, and it shouldn't be a silly pure-Python for for chip. :) These are the goals that one could hope to achieve with an in-place modification that clearly doesn't work.
I am using NumPy 1.8.0.
source to share
I found some discussion after posting this question. An important search term is "slice". Here: http://numpy-discussion.10968.n7.nabble.com/Strange-behavior-in-setting-masked-array-values-in-Numpy-1-1-0-td11999.html
As the day goes on, a discussion appears on the page about trying to detect and warn about it, but it sounds like a lost cause. So I decided to find another method locally to do what I wanted. Here it is!
bc[-1:0:-1] -= bc[-2::-1]
Confirm @fredtantini for explicitly writing a pure-Python loop for a loop that does NumPy efficiently. This got me thinking about how to fix this in pure Python (iterate backwards!), Leading to the above solution.
source to share
I am heading here again to correct my answer. maybe try:
bc-=np.roll( bc , 1 )
#array([-9, 1, 1, 1, 1, 1, 1, 1, 1, 1])
#this arrays length is 10
#and then for an array with length 9
bc=bc[ 1 : ]
#array([1, 1, 1, 1, 1, 1, 1, 1, 1])
Sorry miss-understood the question earlier,
result reason:
'array([0, 1, 1, 2, 2, 3, 3, 4, 4, 5])'
is an
that this action is really syntactically like the output
let (say...) a=array([0,1,2,3,4,5,6,7,8,9])
updated a <- will be formed as [0, 1-0=1,2-1=1,3-1=2,4-2=2,5-2=3,6-3=3,7-3=4,8-4=4,9-4=5]
I think you really want above
tnx EdChum
source to share
I don't think the error is:
By doing bc[1:] -= bc[:-1]
, you are modifying the list as you perform operations.
The process is similar to
for i in range(1,len(bc)):
bc[i] = bc[i] - bc[i-1]
So the next step changes bc[i-1]
:
"i" 1
bc[1:]: 1,2,3,4,5,…
bc[:-1]: 0,1,2,3,4,5,…
bc[1]= 1 - 0 = 1
new bc: 0,1,2,3,4,5,…
"i" 2
bc[1:]: 1,2,3,4,5,…
bc[:-1]: 0,1,2,3,4,5,…
bc[2]= 2 - 1 = 1
new bc: 0,1,1,3,4,5,…
"i" 3
bc[1:]: 1,1,3,4,5,…
bc[:-1]: 0,1,1,3,4,5,…
bc[1]= 3 - 1 = 2
new bc: 0,1,1,2,4,5,…
"i" 4
bc[1:]: 1,1,2,4,5,…
bc[:-1]: 0,1,1,2,4,5,…
bc[1]= 4 - 2 = 2
new bc: 0,1,1,2,2,5,…
Etc.
For the rest of the question, I cannot answer.
source to share