### 最有效的方法来映射函数超过numpy数组。

#### [英]Most efficient way to map function over numpy array

What is the most efficient way to map a function over a numpy array? The way I've been doing it in my current project is as follows:

``````import numpy as np

x = np.array([1, 2, 3, 4, 5])

# Obtain array of square of each element in x
squarer = lambda t: t ** 2
squares = np.array([squarer(xi) for xi in x])
``````

However, this seems like it is probably very inefficient, since I am using a list comprehension to construct the new array as a Python list before converting it back to a numpy array.

Can we do better?

## 8 个解决方案

### #1

63

How about using `numpy.vectorize`.

``````>>> import numpy as np
>>> x = np.array([1, 2, 3, 4, 5])
>>> squarer = lambda t: t ** 2
>>> vfunc = np.vectorize(squarer)
>>> vfunc(x)
array([ 1,  4,  9, 16, 25])
``````

http://docs.scipy.org/doc/numpy-1.10.1/reference/generated/numpy.vectorize.html

http://docs.scipy.org/doc/numpy-1.10.1/reference/generated/numpy.vectorize.html

### #2

59

I've tested all suggested methods plus `np.array(map(f, x))` with `perfplot` (a small project of mine).

Message #1: If you can use numpy's native functions, do that.

If the function you're trying to vectorize already is vectorized (like the `x**2` example in the original post), using that is much faster than anything else (note the log scale):

If you actually need vectorization, it doesn't really matter much which variant you use.

Code to reproduce the plots:

``````import numpy as np
import perfplot
import math

def f(x):
# return math.sqrt(x)
return np.sqrt(x)

vf = np.vectorize(f)

def array_for(x):
return np.array([f(xi) for xi in x])

def array_map(x):
return np.array(list(map(f, x)))

def fromiter(x):
return np.fromiter((f(xi) for xi in x), x.dtype)

def vectorize(x):
return np.vectorize(f)(x)

def vectorize_without_init(x):
return vf(x)

perfplot.show(
setup=lambda n: np.random.rand(n),
n_range=[2**k for k in range(20)],
kernels=[
f,
array_for, array_map, fromiter, vectorize, vectorize_without_init
],
logx=True,
logy=True,
xlabel='len(x)',
)
``````

27

# TL;DR

In my experience, `np.fromiter` is generally the fastest for different sizes of arrays, and different versions of Python and NumPy from different compilers. Here is a short example:

``````import numpy as np
x = np.array([1, 2, 3, 4, 5])
f = lambda x: x ** 2
squares = np.fromiter((f(xi) for xi in x), x.dtype, count=len(x))
``````

Note that `count` is optional. It may be worthwhile to use `np.vectorize` for arrays with at least hundreds of elements (see the plot in Nico's answer).

# Comparison of methods

Here are some simple tests to compare three methods to map a function, this example using with Python 2.7 and NumPy 1.9. First, the set-up functions for testing:

``````import timeit
import numpy as np

f = lambda x: x ** 2
vf = np.vectorize(f)

def test_array(x, n):
t = timeit.timeit(
'np.array([f(xi) for xi in x])',
'from __main__ import np, x, f', number=n)
print('array: ' + str(t))

def test_fromiter(x, n):
t = timeit.timeit(
'np.fromiter((f(xi) for xi in x), x.dtype, count=len(x))',
'from __main__ import np, x, f', number=n)
print('fromiter: ' + str(t))

def test_vectorized(x, n):
t = timeit.timeit(
'vf(x)',
'from __main__ import x, vf', number=n)
print('vectorized: ' + str(t))
``````

With five elements, `np.fromiter` is fastest, and `np.vectorize` is much slower (due to set-up costs):

np.fromiter有五个元素，是最快的，np也是。矢量化速度要慢得多(由于设置成本):

``````x = np.array([1, 2, 3, 4, 5])
n = 100000
test_array(x, n)       # 0.616514921188
test_fromiter(x, n)    # 0.585698843002
test_vectorized(x, n)  # 2.6228120327
``````

With 100s of elements all methods are about the same:

``````x = np.arange(100)
n = 10000
test_array(x, n)       # 0.519502162933
test_fromiter(x, n)    # 0.500586986542
test_vectorized(x, n)  # 0.525988101959
``````

But with 1000s of array elements or more, the vectorized approach is most efficient:

``````x = np.arange(1000)
n = 1000
test_array(x, n)       # 0.472352981567
test_fromiter(x, n)    # 0.453316926956
test_vectorized(x, n)  # 0.291934967041
``````

However, different versions of Python/NumPy and compiler optimization will have different results, so do a similar test for your environment.

### #4

8

``````squares = squarer(x)
``````

Arithmetic operations on arrays are automatically applied elementwise, with efficient C-level loops that avoid all the interpreter overhead that would apply to a Python-level loop or comprehension.

Most of the functions you'd want to apply to a NumPy array elementwise will just work, though some may need changes. For example, `if` doesn't work elementwise. You'd want to convert those to use constructs like `numpy.where`:

``````def using_if(x):
if x < 5:
return x
else:
return x**2
``````

becomes

``````def using_where(x):
return numpy.where(x < 5, x, x**2)
``````

### #5

8

I believe in newer version( I use 1.13) of numpy you can simply call the function by passing the numpy array to the fuction that you wrote for scalar type, it will automatically apply the function call to each element over the numpy array and return you another numpy array

``````>>> import numpy as np
>>> squarer = lambda t: t ** 2
>>> x = np.array([1, 2, 3, 4, 5])
>>> squarer(x)
array([ 1,  4,  9, 16, 25])
``````

### #6

1

As mentioned in this post, just use generator expressions like so:

``````numpy.fromiter((<some_func>(x) for x in <something>),<dtype>,<size of something>)
``````

### #7

0

Maybe this is not directly answering the question, but I've heard that numba can compile existing python code into parallel machine instructions. I'll revisit and revise this post when I actually have a chance to use that.

### #8

-2

Maybe using vectorize is better

``````def square(x):
return x**2

vfunc=vectorize(square)

vfunc([1,2,3,4,5])

output:array([ 1,  4,  9, 16, 25])
``````