Python learnings

Thu, Nov 08 20188 min read

What is a Python generator?

generator, there are two ways to generate generator objects: one is to add parentheses to the list generator:
g1 = (x for x in range(10))
One is to include the ```yield`'' keyword in the function definition:
def fib(max):
n, a, b = 0, 0, 1
while n <max:
yield b
a, b = b, a + b
n = n + 1
g2 = fib(8)
For the generator objects g1 and g2, the value of the next element can be obtained continuously through next(g1). If there are no more elements, an error ```StopIteration`'' will be reported.
You can also get the value of an element through a for loop.
The advantage of the generator is that it doesn't take up a lot of memory, it only needs to calculate the value of the element when it is used.

What is a Python iterator?

Python can be used for for loops, called iterable Iterable, including data structures such as list/set/tuple/str/dict and generators; you can use the following statement to determine whether an object is iterable:
from collections import Iterable
isinstance(x, Iterable)
The iterator Iterator means that it can be called by the next() function and continuously return the next value until StopIteration; generators are all Iterators, and lists, etc. The data structure is not; the list can be turned into an Iterator with the following statement:
Generators are all Iterators, but iterators are not necessarily generators.

What is the difference between list and tuple?

-The length of the list is variable, and the tuple is immutable; -The value of the elements in the list can be changed, but the tuple cannot be changed; -List supports methods such as append; insert; remove; pop, but not tuples

How are list and dict in Python implemented?

Can multiple threads be used in Python to achieve the use of multi-core CPUs together?

There is a thing called Global Interpreter Lock (GIL) in Python that will ensure that only one of your multiple threads is executed at any time. The execution speed of threads is very fast, which will make you mistakenly think that threads are executed in parallel, but in fact they are executed in turns. After the GIL level processing, it will increase the execution overhead.
Multi-core tasks can be achieved through multiple processes.

GIL(Global Interpreter Lock)

Global interpreter lock
The Global Interpreter Lock is a mechanism used by the computer programming language interpreter to synchronize threads. It allows only one thread to execute at any time. Even on multi-core processors, the interpreter that uses GIL only allows one thread to execute at the same time. Common interpreters that use GIL include CPython and Ruby MRI. It can be seen that GIL is not a unique feature of Python, it is a mechanism for an interpreted language to deal with multithreading problems rather than a language feature.

GIL's original intention?

Efficient use of CPU in the single-core era, aiming at interpreter-level data security (not thread-safe thread-safe). The first thing to be clear is that GIL is not a feature of Python, it is a concept introduced when implementing the Python parser (CPython). When the thread of the Python virtual machine wants to call the native thread of C, it needs to know the context of the thread, because there is no way to control the execution of the native thread of C, so the context can only be passed to the native thread. Similarly, the result is also obtained Cheng waits on the python virtual machine. In order to perform a calculation operation, the thread group executing the program must be executed serially.

Why add it to the interpreter instead of other layers?

The GIL lock is added to the interpreter layer, which means that the Cython interpreter that Python calls is added to the GIL lock, because all threads called by your python are native threads. Native thread provides a native interface through C language, which is equivalent to a function of C language. Once you adjust it, you can't control it, and you have to wait for it to return results to you. As long as it has passed the python virtual machine , And further down, it is not controlled by python, it is controlled by the C language itself. It cannot be added under the Python virtual machine, it can only be added at the layer of the Python interpreter.

The implementation of GIL is thread-unsafe? Why?

It is not safe, and the specific circumstances should be discussed separately.
In the case of single core:
  1. When it comes to step 5, python may switch the GIL exactly once (it is said that in python2.7, the GIL will be switched every 100 instructions), the execution time is up, and the GIL is required to be released. At this time thead 1 The count=0 is not executed, but in a suspended state, the context of count=0 is stored in the register.
  2. Then to step 6, at this time thead 2 starts to execute, and then it becomes count = 1, and returns to count, this time count=1.
  3. Then go back to thead 1. At this time, due to the context, count = 0 in the register obtained by thead 1. After calculation, count = 1 is obtained, and the original count = 1 is overwritten by the operation in step 13 Therefore, count is still count = 1 at this time, so this data is not protected.
Both python2.x and 3.x force the GIL to be released when performing IO operations, so that other threads have the opportunity to execute the program.
Python2.x Python uses counter ticks to calculate bytecodes. When 100 bytecodes are executed, GIL is forced to be released, and other threads obtain GIL to continue execution. Ticks can be regarded as Python's own counter, which acts on the GIL specifically, returns to zero after release, and the technology can be adjusted.
Python3.x Python uses a timer. After the execution time reaches the threshold, the current thread releases the GIL. Overall, it is better for CPU-intensive tasks than Python 3.x, but it still does not solve the problem.
In the case of multi-core:
In the case of multiple CPUs, when a single CPU releases the GIL lock, threads on other CPUs will also compete, but CPU-A may get the GIL immediately, so threads on other CPUs can only continue to wait until they return to the waiting state. . In the case of multi-core CPUs, the efficiency of multi-threading will decrease instead, resulting in a lot of waste of resources.

What is a decorator?

Garbage collection mechanism in Python?

What is a lambda expression?

Simply put, lambda expressions are usually used when you need to use a function, but don't want to bother naming a function, which is commonly referred to as an anonymous function.
The general form of lambda expression is: the keyword lambda is followed by one or more parameters, followed by a colon ":", followed by an expression

What are deep copy and shallow copy?

Assignment (=) is to create a new reference to the object. Modifying any one of the variables will affect the other.
Shallow copy copy.copy: Create a new object, but it contains a reference to the items contained in the original object (if one of the objects is modified by reference, the other will also be modified)
Deep copy: create a new object, and recursively copy the objects it contains (modify one, the other will not change) deep.deepcopy() function of the copy module

What is the difference between double equal and is?

== compares the values of two variables, and returns True as long as the values are equal
is compares the id of two variables, that is, id(a) == id(b). Only when two variables point to the same object will it return True
But it should be noted that, such as the following code:
a = 2
b = 2
print(a is b)
According to the above explanation, it should output False, but in fact it will output True. This is because Python has a caching mechanism for small data, and data between -5 and 256 will be cached.

Other Python knowledge points

Type conversion

-list(x) -str(x) -set(x) -int(x) -tuple(x)



-lst[a:b]: left closed and right open -lst.append(value): add elements at the end, complexity O(1) -lst.pop(): pop up the last element of the list, complexity O(1) -lst.pop(index): Pop elements at any position, move the following elements forward, complexity O(n) -lst.insert(index, value): insert an element, move back the following element, complexity O(n) -lst.remove(value): Remove the first element equal to value, and move the following elements forward, and the complexity is O(n) -lst.count(value): the number of elements whose count is value -lst.sort(reverse = False): sort, the default ascending order


Continue with
to comment