pytho****@googl*****
pytho****@googl*****
2011年 3月 20日 (日) 05:36:01 JST
Revision: 6a999dbabfdd Author: Naoki INADA <inada****@klab*****> Date: Sat Mar 19 08:59:41 2011 Log: Add FAQ. http://code.google.com/p/python-doc-ja/source/detail?r=6a999dbabfdd Added: /faq/design.rst /faq/extending.rst /faq/general.rst /faq/gui.rst /faq/index.rst /faq/installed.rst /faq/library.rst /faq/programming.rst /faq/python-video-icon.png /faq/windows.rst ======================================= --- /dev/null +++ /faq/design.rst Sat Mar 19 08:59:41 2011 @@ -0,0 +1,932 @@ +====================== +Design and History FAQ +====================== + +Why does Python use indentation for grouping of statements? +----------------------------------------------------------- + +Guido van Rossum believes that using indentation for grouping is extremely +elegant and contributes a lot to the clarity of the average Python program. +Most people learn to love this feature after a while. + +Since there are no begin/end brackets there cannot be a disagreement between +grouping perceived by the parser and the human reader. Occasionally C +programmers will encounter a fragment of code like this:: + + if (x <= y) + x++; + y--; + z++; + +Only the ``x++`` statement is executed if the condition is true, but the +indentation leads you to believe otherwise. Even experienced C programmers will +sometimes stare at it a long time wondering why ``y`` is being decremented even +for ``x > y``. + +Because there are no begin/end brackets, Python is much less prone to +coding-style conflicts. In C there are many different ways to place the braces. +If you're used to reading and writing code that uses one style, you will feel at +least slightly uneasy when reading (or being required to write) another style. + +Many coding styles place begin/end brackets on a line by themself. This makes +programs considerably longer and wastes valuable screen space, making it harder +to get a good overview of a program. Ideally, a function should fit on one +screen (say, 20-30 lines). 20 lines of Python can do a lot more work than 20 +lines of C. This is not solely due to the lack of begin/end brackets -- the +lack of declarations and the high-level data types are also responsible -- but +the indentation-based syntax certainly helps. + + +Why am I getting strange results with simple arithmetic operations? +------------------------------------------------------------------- + +See the next question. + + +Why are floating point calculations so inaccurate? +-------------------------------------------------- + +People are often very surprised by results like this:: + + >>> 1.2 - 1.0 + 0.199999999999999996 + +and think it is a bug in Python. It's not. This has nothing to do with Python, +but with how the underlying C platform handles floating point numbers, and +ultimately with the inaccuracies introduced when writing down numbers as a +string of a fixed number of digits. + +The internal representation of floating point numbers uses a fixed number of +binary digits to represent a decimal number. Some decimal numbers can't be +represented exactly in binary, resulting in small roundoff errors. + +In decimal math, there are many numbers that can't be represented with a fixed +number of decimal digits, e.g. 1/3 = 0.3333333333....... + +In base 2, 1/2 = 0.1, 1/4 = 0.01, 1/8 = 0.001, etc. .2 equals 2/10 equals 1/5, +resulting in the binary fractional number 0.001100110011001... + +Floating point numbers only have 32 or 64 bits of precision, so the digits are +cut off at some point, and the resulting number is 0.199999999999999996 in +decimal, not 0.2. + +A floating point number's ``repr()`` function prints as many digits are +necessary to make ``eval(repr(f)) == f`` true for any float f. The ``str()`` +function prints fewer digits and this often results in the more sensible number +that was probably intended:: + + >>> 0.2 + 0.20000000000000001 + >>> print 0.2 + 0.2 + +One of the consequences of this is that it is error-prone to compare the result +of some computation to a float with ``==``. Tiny inaccuracies may mean that +``==`` fails. Instead, you have to check that the difference between the two +numbers is less than a certain threshold:: + + epsilon = 0.0000000000001 # Tiny allowed error + expected_result = 0.4 + + if expected_result-epsilon <= computation() <= expected_result+epsilon: + ... + +Please see the chapter on :ref:`floating point arithmetic <tut-fp-issues>` in +the Python tutorial for more information. + + +Why are Python strings immutable? +--------------------------------- + +There are several advantages. + +One is performance: knowing that a string is immutable means we can allocate +space for it at creation time, and the storage requirements are fixed and +unchanging. This is also one of the reasons for the distinction between tuples +and lists. + +Another advantage is that strings in Python are considered as "elemental" as +numbers. No amount of activity will change the value 8 to anything else, and in +Python, no amount of activity will change the string "eight" to anything else. + + +.. _why-self: + +Why must 'self' be used explicitly in method definitions and calls? +------------------------------------------------------------------- + +The idea was borrowed from Modula-3. It turns out to be very useful, for a +variety of reasons. + +First, it's more obvious that you are using a method or instance attribute +instead of a local variable. Reading ``self.x`` or ``self.meth()`` makes it +absolutely clear that an instance variable or method is used even if you don't +know the class definition by heart. In C++, you can sort of tell by the lack of +a local variable declaration (assuming globals are rare or easily recognizable) +-- but in Python, there are no local variable declarations, so you'd have to +look up the class definition to be sure. Some C++ and Java coding standards +call for instance attributes to have an ``m_`` prefix, so this explicitness is +still useful in those languages, too. + +Second, it means that no special syntax is necessary if you want to explicitly +reference or call the method from a particular class. In C++, if you want to +use a method from a base class which is overridden in a derived class, you have +to use the ``::`` operator -- in Python you can write +``baseclass.methodname(self, <argument list>)``. This is particularly useful +for :meth:`__init__` methods, and in general in cases where a derived class +method wants to extend the base class method of the same name and thus has to +call the base class method somehow. + +Finally, for instance variables it solves a syntactic problem with assignment: +since local variables in Python are (by definition!) those variables to which a +value is assigned in a function body (and that aren't explicitly declared +global), there has to be some way to tell the interpreter that an assignment was +meant to assign to an instance variable instead of to a local variable, and it +should preferably be syntactic (for efficiency reasons). C++ does this through +declarations, but Python doesn't have declarations and it would be a pity having +to introduce them just for this purpose. Using the explicit ``self.var`` solves +this nicely. Similarly, for using instance variables, having to write +``self.var`` means that references to unqualified names inside a method don't +have to search the instance's directories. To put it another way, local +variables and instance variables live in two different namespaces, and you need +to tell Python which namespace to use. + + +Why can't I use an assignment in an expression? +----------------------------------------------- + +Many people used to C or Perl complain that they want to use this C idiom: + +.. code-block:: c + + while (line = readline(f)) { + // do something with line + } + +where in Python you're forced to write this:: + + while True: + line = f.readline() + if not line: + break + ... # do something with line + +The reason for not allowing assignment in Python expressions is a common, +hard-to-find bug in those other languages, caused by this construct: + +.. code-block:: c + + if (x = 0) { + // error handling + } + else { + // code that only works for nonzero x + } + +The error is a simple typo: ``x = 0``, which assigns 0 to the variable ``x``, +was written while the comparison ``x == 0`` is certainly what was intended. + +Many alternatives have been proposed. Most are hacks that save some typing but +use arbitrary or cryptic syntax or keywords, and fail the simple criterion for +language change proposals: it should intuitively suggest the proper meaning to a +human reader who has not yet been introduced to the construct. + +An interesting phenomenon is that most experienced Python programmers recognize +the ``while True`` idiom and don't seem to be missing the assignment in +expression construct much; it's only newcomers who express a strong desire to +add this to the language. + +There's an alternative way of spelling this that seems attractive but is +generally less robust than the "while True" solution:: + + line = f.readline() + while line: + ... # do something with line... + line = f.readline() + +The problem with this is that if you change your mind about exactly how you get +the next line (e.g. you want to change it into ``sys.stdin.readline()``) you +have to remember to change two places in your program -- the second occurrence +is hidden at the bottom of the loop. + +The best approach is to use iterators, making it possible to loop through +objects using the ``for`` statement. For example, in the current version of +Python file objects support the iterator protocol, so you can now write simply:: + + for line in f: + ... # do something with line... + + + +Why does Python use methods for some functionality (e.g. list.index()) but functions for other (e.g. len(list))? +---------------------------------------------------------------------------------------------------------------- + +The major reason is history. Functions were used for those operations that were +generic for a group of types and which were intended to work even for objects +that didn't have methods at all (e.g. tuples). It is also convenient to have a +function that can readily be applied to an amorphous collection of objects when +you use the functional features of Python (``map()``, ``apply()`` et al). + +In fact, implementing ``len()``, ``max()``, ``min()`` as a built-in function is +actually less code than implementing them as methods for each type. One can +quibble about individual cases but it's a part of Python, and it's too late to +make such fundamental changes now. The functions have to remain to avoid massive +code breakage. + +.. XXX talk about protocols? + +.. note:: + + For string operations, Python has moved from external functions (the + ``string`` module) to methods. However, ``len()`` is still a function. + + +Why is join() a string method instead of a list or tuple method? +---------------------------------------------------------------- + +Strings became much more like other standard types starting in Python 1.6, when +methods were added which give the same functionality that has always been +available using the functions of the string module. Most of these new methods +have been widely accepted, but the one which appears to make some programmers +feel uncomfortable is:: + + ", ".join(['1', '2', '4', '8', '16']) + +which gives the result:: + + "1, 2, 4, 8, 16" + +There are two common arguments against this usage. + +The first runs along the lines of: "It looks really ugly using a method of a +string literal (string constant)", to which the answer is that it might, but a +string literal is just a fixed value. If the methods are to be allowed on names +bound to strings there is no logical reason to make them unavailable on +literals. + +The second objection is typically cast as: "I am really telling a sequence to +join its members together with a string constant". Sadly, you aren't. For some +reason there seems to be much less difficulty with having :meth:`~str.split` as +a string method, since in that case it is easy to see that :: + + "1, 2, 4, 8, 16".split(", ") + +is an instruction to a string literal to return the substrings delimited by the +given separator (or, by default, arbitrary runs of white space). In this case a +Unicode string returns a list of Unicode strings, an ASCII string returns a list +of ASCII strings, and everyone is happy. + +:meth:`~str.join` is a string method because in using it you are telling the +separator string to iterate over a sequence of strings and insert itself between +adjacent elements. This method can be used with any argument which obeys the +rules for sequence objects, including any new classes you might define yourself. + +Because this is a string method it can work for Unicode strings as well as plain +ASCII strings. If ``join()`` were a method of the sequence types then the +sequence types would have to decide which type of string to return depending on +the type of the separator. + +.. XXX remove next paragraph eventually + +If none of these arguments persuade you, then for the moment you can continue to +use the ``join()`` function from the string module, which allows you to write :: + + string.join(['1', '2', '4', '8', '16'], ", ") + + +How fast are exceptions? +------------------------ + +A try/except block is extremely efficient. Actually catching an exception is +expensive. In versions of Python prior to 2.0 it was common to use this idiom:: + + try: + value = mydict[key] + except KeyError: + mydict[key] = getvalue(key) + value = mydict[key] + +This only made sense when you expected the dict to have the key almost all the +time. If that wasn't the case, you coded it like this:: + + if mydict.has_key(key): + value = mydict[key] + else: + mydict[key] = getvalue(key) + value = mydict[key] + +.. note:: + + In Python 2.0 and higher, you can code this as ``value = + mydict.setdefault(key, getvalue(key))``. + + +Why isn't there a switch or case statement in Python? +----------------------------------------------------- + +You can do this easily enough with a sequence of ``if... elif... elif... else``. +There have been some proposals for switch statement syntax, but there is no +consensus (yet) on whether and how to do range tests. See :pep:`275` for +complete details and the current status. + +For cases where you need to choose from a very large number of possibilities, +you can create a dictionary mapping case values to functions to call. For +example:: + + def function_1(...): + ... + + functions = {'a': function_1, + 'b': function_2, + 'c': self.method_1, ...} + + func = functions[value] + func() + +For calling methods on objects, you can simplify yet further by using the +:func:`getattr` built-in to retrieve methods with a particular name:: + + def visit_a(self, ...): + ... + ... + + def dispatch(self, value): + method_name = 'visit_' + str(value) + method = getattr(self, method_name) + method() + +It's suggested that you use a prefix for the method names, such as ``visit_`` in +this example. Without such a prefix, if values are coming from an untrusted +source, an attacker would be able to call any method on your object. + + +Can't you emulate threads in the interpreter instead of relying on an OS-specific thread implementation? +-------------------------------------------------------------------------------------------------------- + +Answer 1: Unfortunately, the interpreter pushes at least one C stack frame for +each Python stack frame. Also, extensions can call back into Python at almost +random moments. Therefore, a complete threads implementation requires thread +support for C. + +Answer 2: Fortunately, there is `Stackless Python <http://www.stackless.com>`_, +which has a completely redesigned interpreter loop that avoids the C stack. +It's still experimental but looks very promising. Although it is binary +compatible with standard Python, it's still unclear whether Stackless will make +it into the core -- maybe it's just too revolutionary. + + +Why can't lambda forms contain statements? +------------------------------------------ + +Python lambda forms cannot contain statements because Python's syntactic +framework can't handle statements nested inside expressions. However, in +Python, this is not a serious problem. Unlike lambda forms in other languages, +where they add functionality, Python lambdas are only a shorthand notation if +you're too lazy to define a function. + +Functions are already first class objects in Python, and can be declared in a +local scope. Therefore the only advantage of using a lambda form instead of a +locally-defined function is that you don't need to invent a name for the +function -- but that's just a local variable to which the function object (which +is exactly the same type of object that a lambda form yields) is assigned! + + +Can Python be compiled to machine code, C or some other language? +----------------------------------------------------------------- + +Not easily. Python's high level data types, dynamic typing of objects and +run-time invocation of the interpreter (using :func:`eval` or :keyword:`exec`) +together mean that a "compiled" Python program would probably consist mostly of +calls into the Python run-time system, even for seemingly simple operations like +``x+1``. + +Several projects described in the Python newsgroup or at past `Python +conferences <http://python.org/community/workshops/>`_ have shown that this +approach is feasible, although the speedups reached so far are only modest +(e.g. 2x). Jython uses the same strategy for compiling to Java bytecode. (Jim +Hugunin has demonstrated that in combination with whole-program analysis, +speedups of 1000x are feasible for small demo programs. See the proceedings +from the `1997 Python conference +<http://python.org/workshops/1997-10/proceedings/>`_ for more information.) + +Internally, Python source code is always translated into a bytecode +representation, and this bytecode is then executed by the Python virtual +machine. In order to avoid the overhead of repeatedly parsing and translating +modules that rarely change, this byte code is written into a file whose name +ends in ".pyc" whenever a module is parsed. When the corresponding .py file is +changed, it is parsed and translated again and the .pyc file is rewritten. + +There is no performance difference once the .pyc file has been loaded, as the +bytecode read from the .pyc file is exactly the same as the bytecode created by +direct translation. The only difference is that loading code from a .pyc file +is faster than parsing and translating a .py file, so the presence of +precompiled .pyc files improves the start-up time of Python scripts. If +desired, the Lib/compileall.py module can be used to create valid .pyc files for +a given set of modules. + +Note that the main script executed by Python, even if its filename ends in .py, +is not compiled to a .pyc file. It is compiled to bytecode, but the bytecode is +not saved to a file. Usually main scripts are quite short, so this doesn't cost +much speed. + +.. XXX check which of these projects are still alive + +There are also several programs which make it easier to intermingle Python and C +code in various ways to increase performance. See, for example, `Psyco +<http://psyco.sourceforge.net/>`_, `Pyrex +<http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/>`_, `PyInline +<http://pyinline.sourceforge.net/>`_, `Py2Cmod +<http://sourceforge.net/projects/py2cmod/>`_, and `Weave +<http://www.scipy.org/Weave>`_. + + +How does Python manage memory? +------------------------------ + +The details of Python memory management depend on the implementation. The +standard C implementation of Python uses reference counting to detect +inaccessible objects, and another mechanism to collect reference cycles, +periodically executing a cycle detection algorithm which looks for inaccessible +cycles and deletes the objects involved. The :mod:`gc` module provides functions +to perform a garbage collection, obtain debugging statistics, and tune the +collector's parameters. + +Jython relies on the Java runtime so the JVM's garbage collector is used. This +difference can cause some subtle porting problems if your Python code depends on +the behavior of the reference counting implementation. + +.. XXX relevant for Python 2.6? + +Sometimes objects get stuck in tracebacks temporarily and hence are not +deallocated when you might expect. Clear the tracebacks with:: + + import sys + sys.exc_clear() + sys.exc_traceback = sys.last_traceback = None + +Tracebacks are used for reporting errors, implementing debuggers and related +things. They contain a portion of the program state extracted during the +handling of an exception (usually the most recent exception). + +In the absence of circularities and tracebacks, Python programs do not need to +manage memory explicitly. + +Why doesn't Python use a more traditional garbage collection scheme? For one +thing, this is not a C standard feature and hence it's not portable. (Yes, we +know about the Boehm GC library. It has bits of assembler code for *most* +common platforms, not for all of them, and although it is mostly transparent, it +isn't completely transparent; patches are required to get Python to work with +it.) + +Traditional GC also becomes a problem when Python is embedded into other +applications. While in a standalone Python it's fine to replace the standard +malloc() and free() with versions provided by the GC library, an application +embedding Python may want to have its *own* substitute for malloc() and free(), +and may not want Python's. Right now, Python works with anything that +implements malloc() and free() properly. + +In Jython, the following code (which is fine in CPython) will probably run out +of file descriptors long before it runs out of memory:: + + for file in very_long_list_of_files: + f = open(file) + c = f.read(1) + +Using the current reference counting and destructor scheme, each new assignment +to f closes the previous file. Using GC, this is not guaranteed. If you want +to write code that will work with any Python implementation, you should +explicitly close the file or use the :keyword:`with` statement; this will work +regardless of GC:: + + for file in very_long_list_of_files: + with open(file) as f: + c = f.read(1) + + +Why isn't all memory freed when Python exits? +--------------------------------------------- + +Objects referenced from the global namespaces of Python modules are not always +deallocated when Python exits. This may happen if there are circular +references. There are also certain bits of memory that are allocated by the C +library that are impossible to free (e.g. a tool like Purify will complain about +these). Python is, however, aggressive about cleaning up memory on exit and +does try to destroy every single object. + +If you want to force Python to delete certain things on deallocation use the +:mod:`atexit` module to run a function that will force those deletions. + + +Why are there separate tuple and list data types? +------------------------------------------------- + +Lists and tuples, while similar in many respects, are generally used in +fundamentally different ways. Tuples can be thought of as being similar to +Pascal records or C structs; they're small collections of related data which may +be of different types which are operated on as a group. For example, a +Cartesian coordinate is appropriately represented as a tuple of two or three +numbers. + +Lists, on the other hand, are more like arrays in other languages. They tend to +hold a varying number of objects all of which have the same type and which are +operated on one-by-one. For example, ``os.listdir('.')`` returns a list of +strings representing the files in the current directory. Functions which +operate on this output would generally not break if you added another file or +two to the directory. + +Tuples are immutable, meaning that once a tuple has been created, you can't +replace any of its elements with a new value. Lists are mutable, meaning that +you can always change a list's elements. Only immutable elements can be used as +dictionary keys, and hence only tuples and not lists can be used as keys. + + +How are lists implemented? +-------------------------- + +Python's lists are really variable-length arrays, not Lisp-style linked lists. +The implementation uses a contiguous array of references to other objects, and +keeps a pointer to this array and the array's length in a list head structure. + +This makes indexing a list ``a[i]`` an operation whose cost is independent of +the size of the list or the value of the index. + +When items are appended or inserted, the array of references is resized. Some +cleverness is applied to improve the performance of appending items repeatedly; +when the array must be grown, some extra space is allocated so the next few +times don't require an actual resize. + + +How are dictionaries implemented? +--------------------------------- + +Python's dictionaries are implemented as resizable hash tables. Compared to +B-trees, this gives better performance for lookup (the most common operation by +far) under most circumstances, and the implementation is simpler. + +Dictionaries work by computing a hash code for each key stored in the dictionary +using the :func:`hash` built-in function. The hash code varies widely depending +on the key; for example, "Python" hashes to -539294296 while "python", a string +that differs by a single bit, hashes to 1142331976. The hash code is then used +to calculate a location in an internal array where the value will be stored. +Assuming that you're storing keys that all have different hash values, this +means that dictionaries take constant time -- O(1), in computer science notation +-- to retrieve a key. It also means that no sorted order of the keys is +maintained, and traversing the array as the ``.keys()`` and ``.items()`` do will +output the dictionary's content in some arbitrary jumbled order. + + +Why must dictionary keys be immutable? +-------------------------------------- + +The hash table implementation of dictionaries uses a hash value calculated from +the key value to find the key. If the key were a mutable object, its value +could change, and thus its hash could also change. But since whoever changes +the key object can't tell that it was being used as a dictionary key, it can't +move the entry around in the dictionary. Then, when you try to look up the same +object in the dictionary it won't be found because its hash value is different. +If you tried to look up the old value it wouldn't be found either, because the +value of the object found in that hash bin would be different. + +If you want a dictionary indexed with a list, simply convert the list to a tuple +first; the function ``tuple(L)`` creates a tuple with the same entries as the +list ``L``. Tuples are immutable and can therefore be used as dictionary keys. + +Some unacceptable solutions that have been proposed: + +- Hash lists by their address (object ID). This doesn't work because if you + construct a new list with the same value it won't be found; e.g.:: + + mydict = {[1, 2]: '12'} + print mydict[[1, 2]] + + would raise a KeyError exception because the id of the ``[1, 2]`` used in the + second line differs from that in the first line. In other words, dictionary + keys should be compared using ``==``, not using :keyword:`is`. + +- Make a copy when using a list as a key. This doesn't work because the list, + being a mutable object, could contain a reference to itself, and then the + copying code would run into an infinite loop. + +- Allow lists as keys but tell the user not to modify them. This would allow a + class of hard-to-track bugs in programs when you forgot or modified a list by + accident. It also invalidates an important invariant of dictionaries: every + value in ``d.keys()`` is usable as a key of the dictionary. + +- Mark lists as read-only once they are used as a dictionary key. The problem + is that it's not just the top-level object that could change its value; you + could use a tuple containing a list as a key. Entering anything as a key into + a dictionary would require marking all objects reachable from there as + read-only -- and again, self-referential objects could cause an infinite loop. + +There is a trick to get around this if you need to, but use it at your own risk: +You can wrap a mutable structure inside a class instance which has both a +:meth:`__eq__` and a :meth:`__hash__` method. You must then make sure that the +hash value for all such wrapper objects that reside in a dictionary (or other +hash based structure), remain fixed while the object is in the dictionary (or +other structure). :: + + class ListWrapper: + def __init__(self, the_list): + self.the_list = the_list + def __eq__(self, other): + return self.the_list == other.the_list + def __hash__(self): + l = self.the_list + result = 98767 - len(l)*555 + for i, el in enumerate(l): + try: + result = result + (hash(el) % 9999999) * 1001 + i + except Exception: + result = (result % 7777777) + i * 333 + return result + +Note that the hash computation is complicated by the possibility that some +members of the list may be unhashable and also by the possibility of arithmetic +overflow. + +Furthermore it must always be the case that if ``o1 == o2`` (ie ``o1.__eq__(o2) +is True``) then ``hash(o1) == hash(o2)`` (ie, ``o1.__hash__() == o2.__hash__()``), +regardless of whether the object is in a dictionary or not. If you fail to meet +these restrictions dictionaries and other hash based structures will misbehave. + +In the case of ListWrapper, whenever the wrapper object is in a dictionary the +wrapped list must not change to avoid anomalies. Don't do this unless you are +prepared to think hard about the requirements and the consequences of not +meeting them correctly. Consider yourself warned. + + +Why doesn't list.sort() return the sorted list? +----------------------------------------------- + +In situations where performance matters, making a copy of the list just to sort +it would be wasteful. Therefore, :meth:`list.sort` sorts the list in place. In +order to remind you of that fact, it does not return the sorted list. This way, +you won't be fooled into accidentally overwriting a list when you need a sorted +copy but also need to keep the unsorted version around. + +In Python 2.4 a new built-in function -- :func:`sorted` -- has been added. +This function creates a new list from a provided iterable, sorts it and returns +it. For example, here's how to iterate over the keys of a dictionary in sorted +order:: + + for key in sorted(mydict): + ... # do whatever with mydict[key]... + + +How do you specify and enforce an interface spec in Python? +----------------------------------------------------------- + +An interface specification for a module as provided by languages such as C++ and +Java describes the prototypes for the methods and functions of the module. Many +feel that compile-time enforcement of interface specifications helps in the +construction of large programs. + +Python 2.6 adds an :mod:`abc` module that lets you define Abstract Base Classes +(ABCs). You can then use :func:`isinstance` and :func:`issubclass` to check +whether an instance or a class implements a particular ABC. The +:mod:`collections` modules defines a set of useful ABCs such as +:class:`Iterable`, :class:`Container`, and :class:`MutableMapping`. + +For Python, many of the advantages of interface specifications can be obtained +by an appropriate test discipline for components. There is also a tool, +PyChecker, which can be used to find problems due to subclassing. + +A good test suite for a module can both provide a regression test and serve as a +module interface specification and a set of examples. Many Python modules can +be run as a script to provide a simple "self test." Even modules which use +complex external interfaces can often be tested in isolation using trivial +"stub" emulations of the external interface. The :mod:`doctest` and +:mod:`unittest` modules or third-party test frameworks can be used to construct +exhaustive test suites that exercise every line of code in a module. + +An appropriate testing discipline can help build large complex applications in +Python as well as having interface specifications would. In fact, it can be +better because an interface specification cannot test certain properties of a +program. For example, the :meth:`append` method is expected to add new elements +to the end of some internal list; an interface specification cannot test that +your :meth:`append` implementation will actually do this correctly, but it's +trivial to check this property in a test suite. + +Writing test suites is very helpful, and you might want to design your code with +an eye to making it easily tested. One increasingly popular technique, +test-directed development, calls for writing parts of the test suite first, +before you write any of the actual code. Of course Python allows you to be +sloppy and not write test cases at all. + + +Why are default values shared between objects? +---------------------------------------------- + +This type of bug commonly bites neophyte programmers. Consider this function:: + + def foo(mydict={}): # Danger: shared reference to one dict for all calls + ... compute something ... + mydict[key] = value + return mydict + +The first time you call this function, ``mydict`` contains a single item. The +second time, ``mydict`` contains two items because when ``foo()`` begins +executing, ``mydict`` starts out with an item already in it. + +It is often expected that a function call creates new objects for default +values. This is not what happens. Default values are created exactly once, when +the function is defined. If that object is changed, like the dictionary in this +example, subsequent calls to the function will refer to this changed object. + +By definition, immutable objects such as numbers, strings, tuples, and ``None``, +are safe from change. Changes to mutable objects such as dictionaries, lists, +and class instances can lead to confusion. + +Because of this feature, it is good programming practice to not use mutable +objects as default values. Instead, use ``None`` as the default value and +inside the function, check if the parameter is ``None`` and create a new +list/dictionary/whatever if it is. For example, don't write:: + + def foo(mydict={}): + ... + +but:: + + def foo(mydict=None): + if mydict is None: + mydict = {} # create a new dict for local namespace + +This feature can be useful. When you have a function that's time-consuming to +compute, a common technique is to cache the parameters and the resulting value +of each call to the function, and return the cached value if the same value is +requested again. This is called "memoizing", and can be implemented like this:: + + # Callers will never provide a third parameter for this function. + def expensive (arg1, arg2, _cache={}): + if (arg1, arg2) in _cache: + return _cache[(arg1, arg2)] + + # Calculate the value + result = ... expensive computation ... + _cache[(arg1, arg2)] = result # Store result in the cache + return result + +You could use a global variable containing a dictionary instead of the default +value; it's a matter of taste. + + +Why is there no goto? +--------------------- + +You can use exceptions to provide a "structured goto" that even works across +function calls. Many feel that exceptions can conveniently emulate all +reasonable uses of the "go" or "goto" constructs of C, Fortran, and other +languages. For example:: + + class label: pass # declare a label + + try: + ... + if (condition): raise label() # goto label + ... + except label: # where to goto + pass + ... + +This doesn't allow you to jump into the middle of a loop, but that's usually +considered an abuse of goto anyway. Use sparingly. + + +Why can't raw strings (r-strings) end with a backslash? +------------------------------------------------------- + +More precisely, they can't end with an odd number of backslashes: the unpaired +backslash at the end escapes the closing quote character, leaving an +unterminated string. + +Raw strings were designed to ease creating input for processors (chiefly regular +expression engines) that want to do their own backslash escape processing. Such +processors consider an unmatched trailing backslash to be an error anyway, so +raw strings disallow that. In return, they allow you to pass on the string +quote character by escaping it with a backslash. These rules work well when +r-strings are used for their intended purpose. + +If you're trying to build Windows pathnames, note that all Windows system calls +accept forward slashes too:: + + f = open("/mydir/file.txt") # works fine! + +If you're trying to build a pathname for a DOS command, try e.g. one of :: + + dir = r"\this\is\my\dos\dir" "\\" + dir = r"\this\is\my\dos\dir\ "[:-1] + dir = "\\this\\is\\my\\dos\\dir\\" + + +Why doesn't Python have a "with" statement for attribute assignments? +--------------------------------------------------------------------- + +Python has a 'with' statement that wraps the execution of a block, calling code +on the entrance and exit from the block. Some language have a construct that +looks like this:: + + with obj: + a = 1 # equivalent to obj.a = 1 + total = total + 1 # obj.total = obj.total + 1 + +In Python, such a construct would be ambiguous. + +Other languages, such as Object Pascal, Delphi, and C++, use static types, so +it's possible to know, in an unambiguous way, what member is being assigned +to. This is the main point of static typing -- the compiler *always* knows the +scope of every variable at compile time. + +Python uses dynamic types. It is impossible to know in advance which attribute +will be referenced at runtime. Member attributes may be added or removed from +objects on the fly. This makes it impossible to know, from a simple reading, +what attribute is being referenced: a local one, a global one, or a member +attribute? + +For instance, take the following incomplete snippet:: + + def foo(a): + with a: + print x + +The snippet assumes that "a" must have a member attribute called "x". However, +there is nothing in Python that tells the interpreter this. What should happen +if "a" is, let us say, an integer? If there is a global variable named "x", +will it be used inside the with block? As you see, the dynamic nature of Python +makes such choices much harder. + +The primary benefit of "with" and similar language features (reduction of code +volume) can, however, easily be achieved in Python by assignment. Instead of:: + + function(args).mydict[index][index].a = 21 + function(args).mydict[index][index].b = 42 + function(args).mydict[index][index].c = 63 + +write this:: + + ref = function(args).mydict[index][index] + ref.a = 21 + ref.b = 42 + ref.c = 63 + +This also has the side-effect of increasing execution speed because name +bindings are resolved at run-time in Python, and the second version only needs +to perform the resolution once. + + +Why are colons required for the if/while/def/class statements? +-------------------------------------------------------------- + +The colon is required primarily to enhance readability (one of the results of +the experimental ABC language). Consider this:: + + if a == b + print a + +versus :: + + if a == b: + print a + +Notice how the second one is slightly easier to read. Notice further how a +colon sets off the example in this FAQ answer; it's a standard usage in English. + +Another minor reason is that the colon makes it easier for editors with syntax +highlighting; they can look for colons to decide when indentation needs to be +increased instead of having to do a more elaborate parsing of the program text. + + +Why does Python allow commas at the end of lists and tuples? +------------------------------------------------------------ + +Python lets you add a trailing comma at the end of lists, tuples, and +dictionaries:: + + [1, 2, 3,] + ('a', 'b', 'c',) + d = { + "A": [1, 5], + "B": [6, 7], # last trailing comma is optional but good style + } + + +There are several reasons to allow this. + +When you have a literal value for a list, tuple, or dictionary spread across +multiple lines, it's easier to add more elements because you don't have to +remember to add a comma to the previous line. The lines can also be sorted in +your editor without creating a syntax error. + +Accidentally omitting the comma can lead to errors that are hard to diagnose. +For example:: + + x = [ + "fee", + "fie" + "foo", + "fum" + ] + +This list looks like it has four elements, but it actually contains three: +"fee", "fiefoo" and "fum". Always adding the comma avoids this source of error. + +Allowing the trailing comma may also make programmatic code generation easier. ======================================= --- /dev/null +++ /faq/extending.rst Sat Mar 19 08:59:41 2011 @@ -0,0 +1,479 @@ +======================= +Extending/Embedding FAQ +======================= + +.. contents:: + +.. highlight:: c + + +Can I create my own functions in C? +----------------------------------- + +Yes, you can create built-in modules containing functions, variables, exceptions +and even new types in C. This is explained in the document +:ref:`extending-index`. + +Most intermediate or advanced Python books will also cover this topic. + + +Can I create my own functions in C++? +------------------------------------- + +Yes, using the C compatibility features found in C++. Place ``extern "C" { +... }`` around the Python include files and put ``extern "C"`` before each +function that is going to be called by the Python interpreter. Global or static +C++ objects with constructors are probably not a good idea. + + +.. _c-wrapper-software: + +Writing C is hard; are there any alternatives? +---------------------------------------------- + +There are a number of alternatives to writing your own C extensions, depending +on what you're trying to do. + +.. XXX make sure these all work; mention Cython + +If you need more speed, `Psyco <http://psyco.sourceforge.net/>`_ generates x86 +assembly code from Python bytecode. You can use Psyco to compile the most +time-critical functions in your code, and gain a significant improvement with +very little effort, as long as you're running on a machine with an +x86-compatible processor. + +`Pyrex <http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/>`_ is a compiler +that accepts a slightly modified form of Python and generates the corresponding +C code. Pyrex makes it possible to write an extension without having to learn +Python's C API. + +If you need to interface to some C or C++ library for which no Python extension +currently exists, you can try wrapping the library's data types and functions +with a tool such as `SWIG <http://www.swig.org>`_. `SIP +<http://www.riverbankcomputing.co.uk/software/sip/>`__, `CXX +<http://cxx.sourceforge.net/>`_ `Boost +<http://www.boost.org/libs/python/doc/index.html>`_, or `Weave +<http://www.scipy.org/Weave>`_ are also alternatives for wrapping +C++ libraries. + + +How can I execute arbitrary Python statements from C? +----------------------------------------------------- + +The highest-level function to do this is :cfunc:`PyRun_SimpleString` which takes +a single string argument to be executed in the context of the module +``__main__`` and returns 0 for success and -1 when an exception occurred +(including ``SyntaxError``). If you want more control, use +:cfunc:`PyRun_String`; see the source for :cfunc:`PyRun_SimpleString` in +``Python/pythonrun.c``. + + +How can I evaluate an arbitrary Python expression from C? +--------------------------------------------------------- + +Call the function :cfunc:`PyRun_String` from the previous question with the +start symbol :cdata:`Py_eval_input`; it parses an expression, evaluates it and +returns its value. + + +How do I extract C values from a Python object? +----------------------------------------------- + +That depends on the object's type. If it's a tuple, :cfunc:`PyTuple_Size` +returns its length and :cfunc:`PyTuple_GetItem` returns the item at a specified +index. Lists have similar functions, :cfunc:`PyListSize` and +:cfunc:`PyList_GetItem`. + +For strings, :cfunc:`PyString_Size` returns its length and +:cfunc:`PyString_AsString` a pointer to its value. Note that Python strings may +contain null bytes so C's :cfunc:`strlen` should not be used. + +To test the type of an object, first make sure it isn't *NULL*, and then use +:cfunc:`PyString_Check`, :cfunc:`PyTuple_Check`, :cfunc:`PyList_Check`, etc. + +There is also a high-level API to Python objects which is provided by the +so-called 'abstract' interface -- read ``Include/abstract.h`` for further +details. It allows interfacing with any kind of Python sequence using calls +like :cfunc:`PySequence_Length`, :cfunc:`PySequence_GetItem`, etc.) as well as +many other useful protocols. + + +How do I use Py_BuildValue() to create a tuple of arbitrary length? +------------------------------------------------------------------- + +You can't. Use ``t = PyTuple_New(n)`` instead, and fill it with objects using +``PyTuple_SetItem(t, i, o)`` -- note that this "eats" a reference count of +``o``, so you have to :cfunc:`Py_INCREF` it. Lists have similar functions +``PyList_New(n)`` and ``PyList_SetItem(l, i, o)``. Note that you *must* set all +the tuple items to some value before you pass the tuple to Python code -- +``PyTuple_New(n)`` initializes them to NULL, which isn't a valid Python value. + + +How do I call an object's method from C? +---------------------------------------- + +The :cfunc:`PyObject_CallMethod` function can be used to call an arbitrary +method of an object. The parameters are the object, the name of the method to +call, a format string like that used with :cfunc:`Py_BuildValue`, and the +argument values:: + + PyObject * + PyObject_CallMethod(PyObject *object, char *method_name, + char *arg_format, ...); + +This works for any object that has methods -- whether built-in or user-defined. +You are responsible for eventually :cfunc:`Py_DECREF`\ 'ing the return value. + +To call, e.g., a file object's "seek" method with arguments 10, 0 (assuming the +file object pointer is "f"):: + + res = PyObject_CallMethod(f, "seek", "(ii)", 10, 0); + if (res == NULL) { + ... an exception occurred ... + } + else { + Py_DECREF(res); + } + +Note that since :cfunc:`PyObject_CallObject` *always* wants a tuple for the +argument list, to call a function without arguments, pass "()" for the format, +and to call a function with one argument, surround the argument in parentheses, +e.g. "(i)". + + +How do I catch the output from PyErr_Print() (or anything that prints to stdout/stderr)? +---------------------------------------------------------------------------------------- + +In Python code, define an object that supports the ``write()`` method. Assign +this object to :data:`sys.stdout` and :data:`sys.stderr`. Call print_error, or +just allow the standard traceback mechanism to work. Then, the output will go +wherever your ``write()`` method sends it. + +The easiest way to do this is to use the StringIO class in the standard library. + +Sample code and use for catching stdout: + + >>> class StdoutCatcher: + ... def __init__(self): + ... self.data = '' + ... def write(self, stuff): + ... self.data = self.data + stuff + ... + >>> import sys + >>> sys.stdout = StdoutCatcher() + >>> print 'foo' + >>> print 'hello world!' + >>> sys.stderr.write(sys.stdout.data) + foo + hello world! + + +How do I access a module written in Python from C? +-------------------------------------------------- + +You can get a pointer to the module object as follows:: + + module = PyImport_ImportModule("<modulename>"); + +If the module hasn't been imported yet (i.e. it is not yet present in +:data:`sys.modules`), this initializes the module; otherwise it simply returns +the value of ``sys.modules["<modulename>"]``. Note that it doesn't enter the +module into any namespace -- it only ensures it has been initialized and is +stored in :data:`sys.modules`. + +You can then access the module's attributes (i.e. any name defined in the +module) as follows:: + + attr = PyObject_GetAttrString(module, "<attrname>"); + +Calling :cfunc:`PyObject_SetAttrString` to assign to variables in the module +also works. + + +How do I interface to C++ objects from Python? +---------------------------------------------- + +Depending on your requirements, there are many approaches. To do this manually, +begin by reading :ref:`the "Extending and Embedding" document +<extending-index>`. Realize that for the Python run-time system, there isn't a +whole lot of difference between C and C++ -- so the strategy of building a new +Python type around a C structure (pointer) type will also work for C++ objects. + +For C++ libraries, see :ref:`c-wrapper-software`. + + +I added a module using the Setup file and the make fails; why? +-------------------------------------------------------------- + +Setup must end in a newline, if there is no newline there, the build process +fails. (Fixing this requires some ugly shell script hackery, and this bug is so +minor that it doesn't seem worth the effort.) + + +How do I debug an extension? +---------------------------- + +When using GDB with dynamically loaded extensions, you can't set a breakpoint in +your extension until your extension is loaded. + +In your ``.gdbinit`` file (or interactively), add the command:: + + br _PyImport_LoadDynamicModule + +Then, when you run GDB:: + + $ gdb /local/bin/python + gdb) run myscript.py + gdb) continue # repeat until your extension is loaded + gdb) finish # so that your extension is loaded + gdb) br myfunction.c:50 + gdb) continue + +I want to compile a Python module on my Linux system, but some files are missing. Why? +-------------------------------------------------------------------------------------- + +Most packaged versions of Python don't include the +:file:`/usr/lib/python2.{x}/config/` directory, which contains various files +required for compiling Python extensions. + +For Red Hat, install the python-devel RPM to get the necessary files. + +For Debian, run ``apt-get install python-dev``. + + +What does "SystemError: _PyImport_FixupExtension: module yourmodule not loaded" mean? +------------------------------------------------------------------------------------- + +This means that you have created an extension module named "yourmodule", but +your module init function does not initialize with that name. + +Every module init function will have a line similar to:: + + module = Py_InitModule("yourmodule", yourmodule_functions); + +If the string passed to this function is not the same name as your extension +module, the :exc:`SystemError` exception will be raised. + + +How do I tell "incomplete input" from "invalid input"? +------------------------------------------------------ + +Sometimes you want to emulate the Python interactive interpreter's behavior, +where it gives you a continuation prompt when the input is incomplete (e.g. you +typed the start of an "if" statement or you didn't close your parentheses or +triple string quotes), but it gives you a syntax error message immediately when +the input is invalid. + +In Python you can use the :mod:`codeop` module, which approximates the parser's +behavior sufficiently. IDLE uses this, for example. + +The easiest way to do it in C is to call :cfunc:`PyRun_InteractiveLoop` (perhaps +in a separate thread) and let the Python interpreter handle the input for +you. You can also set the :cfunc:`PyOS_ReadlineFunctionPointer` to point at your +custom input function. See ``Modules/readline.c`` and ``Parser/myreadline.c`` +for more hints. + +However sometimes you have to run the embedded Python interpreter in the same +thread as your rest application and you can't allow the +:cfunc:`PyRun_InteractiveLoop` to stop while waiting for user input. The one +solution then is to call :cfunc:`PyParser_ParseString` and test for ``e.error`` +equal to ``E_EOF``, which means the input is incomplete). Here's a sample code +fragment, untested, inspired by code from Alex Farber:: + + #include <Python.h> + #include <node.h> + #include <errcode.h> + #include <grammar.h> + #include <parsetok.h> + #include <compile.h> + + int testcomplete(char *code) + /* code should end in \n */ + /* return -1 for error, 0 for incomplete, 1 for complete */ + { + node *n; + perrdetail e; + + n = PyParser_ParseString(code, &_PyParser_Grammar, + Py_file_input, &e); + if (n == NULL) { + if (e.error == E_EOF) + return 0; + return -1; + } + + PyNode_Free(n); + return 1; + } + +Another solution is trying to compile the received string with +:cfunc:`Py_CompileString`. If it compiles without errors, try to execute the +returned code object by calling :cfunc:`PyEval_EvalCode`. Otherwise save the +input for later. If the compilation fails, find out if it's an error or just +more input is required - by extracting the message string from the exception +tuple and comparing it to the string "unexpected EOF while parsing". Here is a +complete example using the GNU readline library (you may want to ignore +**SIGINT** while calling readline()):: + + #include <stdio.h> + #include <readline.h> + + #include <Python.h> + #include <object.h> + #include <compile.h> + #include <eval.h> + + int main (int argc, char* argv[]) + { + int i, j, done = 0; /* lengths of line, code */ + char ps1[] = ">>> "; + char ps2[] = "... "; + char *prompt = ps1; + char *msg, *line, *code = NULL; + PyObject *src, *glb, *loc; + PyObject *exc, *val, *trb, *obj, *dum; + + Py_Initialize (); + loc = PyDict_New (); + glb = PyDict_New (); + PyDict_SetItemString (glb, "__builtins__", PyEval_GetBuiltins ()); + + while (!done) + { + line = readline (prompt); + + if (NULL == line) /* CTRL-D pressed */ + { + done = 1; + } + else + { + i = strlen (line); + + if (i > 0) + add_history (line); /* save non-empty lines */ + + if (NULL == code) /* nothing in code yet */ + j = 0; + else + j = strlen (code); + + code = realloc (code, i + j + 2); + if (NULL == code) /* out of memory */ + exit (1); + + if (0 == j) /* code was empty, so */ + code[0] = '\0'; /* keep strncat happy */ + + strncat (code, line, i); /* append line to code */ + code[i + j] = '\n'; /* append '\n' to code */ + code[i + j + 1] = '\0'; + + src = Py_CompileString (code, "<stdin>", Py_single_input); + + if (NULL != src) /* compiled just fine - */ + { + if (ps1 == prompt || /* ">>> " or */ + '\n' == code[i + j - 1]) /* "... " and double '\n' */ + { /* so execute it */ + dum = PyEval_EvalCode ((PyCodeObject *)src, glb, loc); + Py_XDECREF (dum); + Py_XDECREF (src); + free (code); + code = NULL; + if (PyErr_Occurred ()) + PyErr_Print (); + prompt = ps1; + } + } /* syntax error or E_EOF? */ + else if (PyErr_ExceptionMatches (PyExc_SyntaxError)) + { + PyErr_Fetch (&exc, &val, &trb); /* clears exception! */ + + if (PyArg_ParseTuple (val, "sO", &msg, &obj) && + !strcmp (msg, "unexpected EOF while parsing")) /* E_EOF */ + { + Py_XDECREF (exc); + Py_XDECREF (val); + Py_XDECREF (trb); + prompt = ps2; + } + else /* some other syntax error */ + { + PyErr_Restore (exc, val, trb); + PyErr_Print (); + free (code); + code = NULL; + prompt = ps1; + } + } + else /* some non-syntax error */ + { + PyErr_Print (); + free (code); + code = NULL; + prompt = ps1; + } + + free (line); + } + } + + Py_XDECREF(glb); + Py_XDECREF(loc); + Py_Finalize(); + exit(0); + } + + +How do I find undefined g++ symbols __builtin_new or __pure_virtual? +-------------------------------------------------------------------- + +To dynamically load g++ extension modules, you must recompile Python, relink it +using g++ (change LINKCC in the Python Modules Makefile), and link your +extension module using g++ (e.g., ``g++ -shared -o mymodule.so mymodule.o``). + + +Can I create an object class with some methods implemented in C and others in Python (e.g. through inheritance)? +---------------------------------------------------------------------------------------------------------------- + +In Python 2.2, you can inherit from built-in classes such as :class:`int`, +:class:`list`, :class:`dict`, etc. + +The Boost Python Library (BPL, http://www.boost.org/libs/python/doc/index.html) +provides a way of doing this from C++ (i.e. you can inherit from an extension +class written in C++ using the BPL). + + +When importing module X, why do I get "undefined symbol: PyUnicodeUCS2*"? +------------------------------------------------------------------------- + +You are using a version of Python that uses a 4-byte representation for Unicode +characters, but some C extension module you are importing was compiled using a +Python that uses a 2-byte representation for Unicode characters (the default). + +If instead the name of the undefined symbol starts with ``PyUnicodeUCS4``, the +problem is the reverse: Python was built using 2-byte Unicode characters, and +the extension module was compiled using a Python with 4-byte Unicode characters. + +This can easily occur when using pre-built extension packages. RedHat Linux +7.x, in particular, provided a "python2" binary that is compiled with 4-byte +Unicode. This only causes the link failure if the extension uses any of the +``PyUnicode_*()`` functions. It is also a problem if an extension uses any of +the Unicode-related format specifiers for :cfunc:`Py_BuildValue` (or similar) or +parameter specifications for :cfunc:`PyArg_ParseTuple`. + +You can check the size of the Unicode character a Python interpreter is using by +checking the value of sys.maxunicode: + + >>> import sys + >>> if sys.maxunicode > 65535: + ... print 'UCS4 build' + ... else: + ... print 'UCS2 build' + +The only way to solve this problem is to use extension modules compiled with a +Python binary built using the same size for Unicode characters. + + + ======================================= --- /dev/null +++ /faq/general.rst Sat Mar 19 08:59:41 2011 @@ -0,0 +1,512 @@ +:tocdepth: 2 + +================== +General Python FAQ +================== + +.. contents:: + +General Information +=================== + +What is Python? +--------------- + +Python is an interpreted, interactive, object-oriented programming language. It +incorporates modules, exceptions, dynamic typing, very high level dynamic data +types, and classes. Python combines remarkable power with very clear syntax. +It has interfaces to many system calls and libraries, as well as to various +window systems, and is extensible in C or C++. It is also usable as an +extension language for applications that need a programmable interface. +Finally, Python is portable: it runs on many Unix variants, on the Mac, and on +PCs under MS-DOS, Windows, Windows NT, and OS/2. + +To find out more, start with :ref:`tutorial-index`. The `Beginner's Guide to +Python <http://wiki.python.org/moin/BeginnersGuide>`_ links to other +introductory tutorials and resources for learning Python. + + +What is the Python Software Foundation? +--------------------------------------- + +The Python Software Foundation is an independent non-profit organization that +holds the copyright on Python versions 2.1 and newer. The PSF's mission is to +advance open source technology related to the Python programming language and to +publicize the use of Python. The PSF's home page is at +http://www.python.org/psf/. + +Donations to the PSF are tax-exempt in the US. If you use Python and find it +helpful, please contribute via `the PSF donation page +<http://www.python.org/psf/donations/>`_. + + +Are there copyright restrictions on the use of Python? +------------------------------------------------------ + +You can do anything you want with the source, as long as you leave the +copyrights in and display those copyrights in any documentation about Python +that you produce. If you honor the copyright rules, it's OK to use Python for +commercial use, to sell copies of Python in source or binary form (modified or +unmodified), or to sell products that incorporate Python in some form. We would +still like to know about all commercial use of Python, of course. + +See `the PSF license page <http://python.org/psf/license/>`_ to find further +explanations and a link to the full text of the license. + +The Python logo is trademarked, and in certain cases permission is required to +use it. Consult `the Trademark Usage Policy +<http://www.python.org/psf/trademarks/>`__ for more information. + + +Why was Python created in the first place? +------------------------------------------ + +Here's a *very* brief summary of what started it all, written by Guido van +Rossum: + + I had extensive experience with implementing an interpreted language in the + ABC group at CWI, and from working with this group I had learned a lot about + language design. This is the origin of many Python features, including the + use of indentation for statement grouping and the inclusion of + very-high-level data types (although the details are all different in + Python). + + I had a number of gripes about the ABC language, but also liked many of its + features. It was impossible to extend the ABC language (or its + implementation) to remedy my complaints -- in fact its lack of extensibility + was one of its biggest problems. I had some experience with using Modula-2+ + and talked with the designers of Modula-3 and read the Modula-3 report. + Modula-3 is the origin of the syntax and semantics used for exceptions, and + some other Python features. + + I was working in the Amoeba distributed operating system group at CWI. We + needed a better way to do system administration than by writing either C + programs or Bourne shell scripts, since Amoeba had its own system call + interface which wasn't easily accessible from the Bourne shell. My + experience with error handling in Amoeba made me acutely aware of the + importance of exceptions as a programming language feature. + + It occurred to me that a scripting language with a syntax like ABC but with + access to the Amoeba system calls would fill the need. I realized that it + would be foolish to write an Amoeba-specific language, so I decided that I + needed a language that was generally extensible. + + During the 1989 Christmas holidays, I had a lot of time on my hand, so I + decided to give it a try. During the next year, while still mostly working + on it in my own time, Python was used in the Amoeba project with increasing + success, and the feedback from colleagues made me add many early + improvements. + + In February 1991, after just over a year of development, I decided to post to + USENET. The rest is in the ``Misc/HISTORY`` file. + + +What is Python good for? +------------------------ + +Python is a high-level general-purpose programming language that can be applied +to many different classes of problems. + +The language comes with a large standard library that covers areas such as +string processing (regular expressions, Unicode, calculating differences between +files), Internet protocols (HTTP, FTP, SMTP, XML-RPC, POP, IMAP, CGI +programming), software engineering (unit testing, logging, profiling, parsing +Python code), and operating system interfaces (system calls, filesystems, TCP/IP +sockets). Look at the table of contents for :ref:`library-index` to get an idea +of what's available. A wide variety of third-party extensions are also +available. Consult `the Python Package Index <http://pypi.python.org/pypi>`_ to +find packages of interest to you. + + +How does the Python version numbering scheme work? +-------------------------------------------------- + +Python versions are numbered A.B.C or A.B. A is the major version number -- it +is only incremented for really major changes in the language. B is the minor +version number, incremented for less earth-shattering changes. C is the +micro-level -- it is incremented for each bugfix release. See :pep:`6` for more +information about bugfix releases. + +Not all releases are bugfix releases. In the run-up to a new major release, a +series of development releases are made, denoted as alpha, beta, or release +candidate. Alphas are early releases in which interfaces aren't yet finalized; +it's not unexpected to see an interface change between two alpha releases. +Betas are more stable, preserving existing interfaces but possibly adding new +modules, and release candidates are frozen, making no changes except as needed +to fix critical bugs. + +Alpha, beta and release candidate versions have an additional suffix. The +suffix for an alpha version is "aN" for some small number N, the suffix for a +beta version is "bN" for some small number N, and the suffix for a release +candidate version is "cN" for some small number N. In other words, all versions +labeled 2.0aN precede the versions labeled 2.0bN, which precede versions labeled +2.0cN, and *those* precede 2.0. + +You may also find version numbers with a "+" suffix, e.g. "2.2+". These are +unreleased versions, built directly from the Subversion trunk. In practice, +after a final minor release is made, the Subversion trunk is incremented to the +next minor version, which becomes the "a0" version, +e.g. "2.4a0". + +See also the documentation for ``sys.version``, ``sys.hexversion``, and +``sys.version_info``. + + +How do I obtain a copy of the Python source? +-------------------------------------------- + +The latest Python source distribution is always available from python.org, at +http://www.python.org/download/. The latest development sources can be obtained +via anonymous Subversion at http://svn.python.org/projects/python/trunk. + +The source distribution is a gzipped tar file containing the complete C source, +Sphinx-formatted documentation, Python library modules, example programs, and +several useful pieces of freely distributable software. The source will compile +and run out of the box on most UNIX platforms. + +.. XXX update link once the dev faq is relocated + +Consult the `Developer FAQ <http://www.python.org/dev/faq/>`__ for more +information on getting the source code and compiling it. + + +How do I get documentation on Python? +------------------------------------- + +.. XXX mention py3k + +The standard documentation for the current stable version of Python is available +at http://docs.python.org/. PDF, plain text, and downloadable HTML versions are +also available at http://docs.python.org/download.html. + +The documentation is written in reStructuredText and processed by `the Sphinx +documentation tool <http://sphinx.pocoo.org/>`__. The reStructuredText source +for the documentation is part of the Python source distribution. + + +I've never programmed before. Is there a Python tutorial? +--------------------------------------------------------- + +There are numerous tutorials and books available. The standard documentation +includes :ref:`tutorial-index`. + +Consult `the Beginner's Guide <http://wiki.python.org/moin/BeginnersGuide>`_ to +find information for beginning Python programmers, including lists of tutorials. + + +Is there a newsgroup or mailing list devoted to Python? +------------------------------------------------------- + +There is a newsgroup, :newsgroup:`comp.lang.python`, and a mailing list, +`python-list <http://mail.python.org/mailman/listinfo/python-list>`_. The +newsgroup and mailing list are gatewayed into each other -- if you can read news +it's unnecessary to subscribe to the mailing list. +:newsgroup:`comp.lang.python` is high-traffic, receiving hundreds of postings +every day, and Usenet readers are often more able to cope with this volume. + +Announcements of new software releases and events can be found in +comp.lang.python.announce, a low-traffic moderated list that receives about five +postings per day. It's available as `the python-announce mailing list +<http://mail.python.org/mailman/listinfo/python-announce-list>`_. + +More info about other mailing lists and newsgroups +can be found at http://www.python.org/community/lists/. + + +How do I get a beta test version of Python? +------------------------------------------- + +Alpha and beta releases are available from http://www.python.org/download/. All +releases are announced on the comp.lang.python and comp.lang.python.announce +newsgroups and on the Python home page at http://www.python.org/; an RSS feed of +news is available. + +.. XXX update link once the dev faq is relocated + +You can also access the development version of Python through Subversion. See +http://www.python.org/dev/faq/ for details. + + +How do I submit bug reports and patches for Python? +--------------------------------------------------- + +To report a bug or submit a patch, please use the Roundup installation at +http://bugs.python.org/. + +You must have a Roundup account to report bugs; this makes it possible for us to +contact you if we have follow-up questions. It will also enable Roundup to send +you updates as we act on your bug. If you had previously used SourceForge to +report bugs to Python, you can obtain your Roundup password through Roundup's +`password reset procedure <http://bugs.python.org/user?@template=forgotten>`_. + +.. XXX adapt link to dev guide + +For more information on how Python is developed, consult `the Python Developer's +Guide <http://python.org/dev/>`_. + + +Are there any published articles about Python that I can reference? +------------------------------------------------------------------- + +It's probably best to cite your favorite book about Python. + +The very first article about Python was written in 1991 and is now quite +outdated. + + Guido van Rossum and Jelke de Boer, "Interactively Testing Remote Servers + Using the Python Programming Language", CWI Quarterly, Volume 4, Issue 4 + (December 1991), Amsterdam, pp 283-303. + + +Are there any books on Python? +------------------------------ + +Yes, there are many, and more are being published. See the python.org wiki at +http://wiki.python.org/moin/PythonBooks for a list. + +You can also search online bookstores for "Python" and filter out the Monty +Python references; or perhaps search for "Python" and "language". + + +Where in the world is www.python.org located? +--------------------------------------------- + +It's currently in Amsterdam, graciously hosted by `XS4ALL +<http://www.xs4all.nl>`_. Thanks to Thomas Wouters for his work in arranging +python.org's hosting. + + +Why is it called Python? +------------------------ + +When he began implementing Python, Guido van Rossum was also reading the +published scripts from `"Monty Python's Flying Circus" +<http://pythonline.com/>`__, a BBC comedy series from the 1970s. Van Rossum +thought he needed a name that was short, unique, and slightly mysterious, so he +decided to call the language Python. + + +Do I have to like "Monty Python's Flying Circus"? +------------------------------------------------- + +No, but it helps. :) + + +Python in the real world +======================== + +How stable is Python? +--------------------- + +Very stable. New, stable releases have been coming out roughly every 6 to 18 +months since 1991, and this seems likely to continue. Currently there are +usually around 18 months between major releases. + +The developers issue "bugfix" releases of older versions, so the stability of +existing releases gradually improves. Bugfix releases, indicated by a third +component of the version number (e.g. 2.5.3, 2.6.2), are managed for stability; +only fixes for known problems are included in a bugfix release, and it's +guaranteed that interfaces will remain the same throughout a series of bugfix +releases. + +The latest stable releases can always be found on the `Python download page +<http://python.org/download/>`_. There are two recommended production-ready +versions at this point in time, because at the moment there are two branches of +stable releases: 2.x and 3.x. Python 3.x may be less useful than 2.x, since +currently there is more third party software available for Python 2 than for +Python 3. Python 2 code will generally not run unchanged in Python 3. + + +How many people are using Python? +--------------------------------- + +There are probably tens of thousands of users, though it's difficult to obtain +an exact count. + +Python is available for free download, so there are no sales figures, and it's +available from many different sites and packaged with many Linux distributions, +so download statistics don't tell the whole story either. + +The comp.lang.python newsgroup is very active, but not all Python users post to +the group or even read it. + + +Have any significant projects been done in Python? +-------------------------------------------------- + +See http://python.org/about/success for a list of projects that use Python. +Consulting the proceedings for `past Python conferences +<http://python.org/community/workshops/>`_ will reveal contributions from many +different companies and organizations. + +High-profile Python projects include `the Mailman mailing list manager +<http://www.list.org>`_ and `the Zope application server +<http://www.zope.org>`_. Several Linux distributions, most notably `Red Hat +<http://www.redhat.com>`_, have written part or all of their installer and +system administration software in Python. Companies that use Python internally +include Google, Yahoo, and Lucasfilm Ltd. + + +What new developments are expected for Python in the future? +------------------------------------------------------------ + +See http://www.python.org/dev/peps/ for the Python Enhancement Proposals +(PEPs). PEPs are design documents describing a suggested new feature for Python, +providing a concise technical specification and a rationale. Look for a PEP +titled "Python X.Y Release Schedule", where X.Y is a version that hasn't been +publicly released yet. + +New development is discussed on `the python-dev mailing list +<http://mail.python.org/mailman/listinfo/python-dev/>`_. + + +Is it reasonable to propose incompatible changes to Python? +----------------------------------------------------------- + +In general, no. There are already millions of lines of Python code around the +world, so any change in the language that invalidates more than a very small +fraction of existing programs has to be frowned upon. Even if you can provide a +conversion program, there's still the problem of updating all documentation; +many books have been written about Python, and we don't want to invalidate them +all at a single stroke. + +Providing a gradual upgrade path is necessary if a feature has to be changed. +:pep:`5` describes the procedure followed for introducing backward-incompatible +changes while minimizing disruption for users. + + +Is Python Y2K (Year 2000) Compliant? +------------------------------------ + +.. remove this question? + +As of August, 2003 no major problems have been reported and Y2K compliance seems +to be a non-issue. + +Python does very few date calculations and for those it does perform relies on +the C library functions. Python generally represents times either as seconds +since 1970 or as a ``(year, month, day, ...)`` tuple where the year is expressed +with four digits, which makes Y2K bugs unlikely. So as long as your C library +is okay, Python should be okay. Of course, it's possible that a particular +application written in Python makes assumptions about 2-digit years. + +Because Python is available free of charge, there are no absolute guarantees. +If there *are* unforeseen problems, liability is the user's problem rather than +the developers', and there is nobody you can sue for damages. The Python +copyright notice contains the following disclaimer: + + 4. PSF is making Python 2.3 available to Licensee on an "AS IS" + basis. PSF MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR IMPLIED. BY + WAY OF EXAMPLE, BUT NOT LIMITATION, PSF MAKES NO AND DISCLAIMS ANY + REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS FOR ANY PARTICULAR + PURPOSE OR THAT THE USE OF PYTHON 2.3 WILL NOT INFRINGE ANY THIRD PARTY + RIGHTS. + + 5. PSF SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF PYTHON + 2.3 FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS AS + A RESULT OF MODIFYING, DISTRIBUTING, OR OTHERWISE USING PYTHON 2.3, + OR ANY DERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF. + +The good news is that *if* you encounter a problem, you have full source +available to track it down and fix it. This is one advantage of an open source +programming environment. + + +Is Python a good language for beginning programmers? +---------------------------------------------------- + +Yes. + +It is still common to start students with a procedural and statically typed +language such as Pascal, C, or a subset of C++ or Java. Students may be better +served by learning Python as their first language. Python has a very simple and +consistent syntax and a large standard library and, most importantly, using +Python in a beginning programming course lets students concentrate on important +programming skills such as problem decomposition and data type design. With +Python, students can be quickly introduced to basic concepts such as loops and +procedures. They can probably even work with user-defined objects in their very +first course. + +For a student who has never programmed before, using a statically typed language +seems unnatural. It presents additional complexity that the student must master +and slows the pace of the course. The students are trying to learn to think +like a computer, decompose problems, design consistent interfaces, and +encapsulate data. While learning to use a statically typed language is +important in the long term, it is not necessarily the best topic to address in +the students' first programming course. + +Many other aspects of Python make it a good first language. Like Java, Python +has a large standard library so that students can be assigned programming +projects very early in the course that *do* something. Assignments aren't +restricted to the standard four-function calculator and check balancing +programs. By using the standard library, students can gain the satisfaction of +working on realistic applications as they learn the fundamentals of programming. +Using the standard library also teaches students about code reuse. Third-party +modules such as PyGame are also helpful in extending the students' reach. + +Python's interactive interpreter enables students to test language features +while they're programming. They can keep a window with the interpreter running +while they enter their program's source in another window. If they can't +remember the methods for a list, they can do something like this:: + + >>> L = [] + >>> dir(L) + ['append', 'count', 'extend', 'index', 'insert', 'pop', 'remove', + 'reverse', 'sort'] + >>> help(L.append) + Help on built-in function append: + + append(...) + L.append(object) -- append object to end + >>> L.append(1) + >>> L + [1] + +With the interpreter, documentation is never far from the student as he's +programming. + +There are also good IDEs for Python. IDLE is a cross-platform IDE for Python +that is written in Python using Tkinter. PythonWin is a Windows-specific IDE. +Emacs users will be happy to know that there is a very good Python mode for +Emacs. All of these programming environments provide syntax highlighting, +auto-indenting, and access to the interactive interpreter while coding. Consult +http://www.python.org/editors/ for a full list of Python editing environments. + +If you want to discuss Python's use in education, you may be interested in +joining `the edu-sig mailing list +<http://python.org/community/sigs/current/edu-sig>`_. + + +Upgrading Python +================ + +What is this bsddb185 module my application keeps complaining about? +-------------------------------------------------------------------- + +.. XXX remove this question? + +Starting with Python2.3, the distribution includes the `PyBSDDB package +<http://pybsddb.sf.net/>` as a replacement for the old bsddb module. It +includes functions which provide backward compatibility at the API level, but +requires a newer version of the underlying `Berkeley DB +<http://www.sleepycat.com>`_ library. Files created with the older bsddb module +can't be opened directly using the new module. + +Using your old version of Python and a pair of scripts which are part of Python +2.3 (db2pickle.py and pickle2db.py, in the Tools/scripts directory) you can +convert your old database files to the new format. Using your old Python +version, run the db2pickle.py script to convert it to a pickle, e.g.:: + + python2.2 <pathto>/db2pickley.py database.db database.pck + +Rename your database file:: + + mv database.db olddatabase.db + +Now convert the pickle file to a new format database:: + + python <pathto>/pickle2db.py database.db database.pck + +The precise commands you use will vary depending on the particulars of your +installation. For full details about operation of these two scripts check the +doc string at the start of each one. ======================================= --- /dev/null +++ /faq/gui.rst Sat Mar 19 08:59:41 2011 @@ -0,0 +1,164 @@ +:tocdepth: 2 + +========================== +Graphic User Interface FAQ +========================== + +.. contents:: + +General GUI Questions +===================== + +What platform-independent GUI toolkits exist for Python? +-------------------------------------------------------- + +Depending on what platform(s) you are aiming at, there are several. + +.. XXX check links + +Tkinter +''''''' + +Standard builds of Python include an object-oriented interface to the Tcl/Tk +widget set, called Tkinter. This is probably the easiest to install and use. +For more info about Tk, including pointers to the source, see the Tcl/Tk home +page at http://www.tcl.tk. Tcl/Tk is fully portable to the MacOS, Windows, and +Unix platforms. + +wxWidgets +''''''''' + +wxWidgets is a GUI class library written in C++ that's a portable +interface to various platform-specific libraries, and that has a +Python interface called `wxPython <http://www.wxpython.org>`__. + +wxWidgets preserves the look and feel of the +underlying graphics toolkit, and has a large set of widgets and +collection of GDI classes. See `the wxWidgets page +<http://www.wxwidgets.org>`_ for more details. + +wxWidgets supports Windows and MacOS; on Unix variants, +it supports both GTk+ and Motif toolkits. + +Qt +''' + +There are bindings available for the Qt toolkit (`PyQt +<http://www.riverbankcomputing.co.uk/software/pyqt/>`_) and for KDE (`PyKDE <http://www.riverbankcomputing.co.uk/software/pykde/intro>`__). If +you're writing open source software, you don't need to pay for PyQt, but if you +want to write proprietary applications, you must buy a PyQt license from +`Riverbank Computing <http://www.riverbankcomputing.co.uk>`_ and (up to Qt 4.4; +Qt 4.5 upwards is licensed under the LGPL license) a Qt license from `Trolltech +<http://www.trolltech.com>`_. + +Gtk+ +'''' + +PyGtk bindings for the `Gtk+ toolkit <http://www.gtk.org>`_ have been +implemented by James Henstridge; see <http://www.pygtk.org>. + +FLTK +'''' + +Python bindings for `the FLTK toolkit <http://www.fltk.org>`_, a simple yet +powerful and mature cross-platform windowing system, are available from `the +PyFLTK project <http://pyfltk.sourceforge.net>`_. + + +FOX +''' + +A wrapper for `the FOX toolkit <http://www.fox-toolkit.org/>`_ called `FXpy +<http://fxpy.sourceforge.net/>`_ is available. FOX supports both Unix variants +and Windows. + + +OpenGL +'''''' + +For OpenGL bindings, see `PyOpenGL <http://pyopengl.sourceforge.net>`_. + + +What platform-specific GUI toolkits exist for Python? +----------------------------------------------------- + +`The Mac port <http://python.org/download/mac>`_ by Jack Jansen has a rich and +ever-growing set of modules that support the native Mac toolbox calls. The port +supports MacOS X's Carbon libraries. + +By installing the `PyObjc Objective-C bridge +<http://pyobjc.sourceforge.net>`_, Python programs can use MacOS X's +Cocoa libraries. See the documentation that comes with the Mac port. + +:ref:`Pythonwin <windows-faq>` by Mark Hammond includes an interface to the +Microsoft Foundation Classes and a Python programming environment +that's written mostly in Python using the MFC classes. + + +Tkinter questions +================= + +How do I freeze Tkinter applications? +------------------------------------- + +Freeze is a tool to create stand-alone applications. When freezing Tkinter +applications, the applications will not be truly stand-alone, as the application +will still need the Tcl and Tk libraries. + +One solution is to ship the application with the Tcl and Tk libraries, and point +to them at run-time using the :envvar:`TCL_LIBRARY` and :envvar:`TK_LIBRARY` +environment variables. + +To get truly stand-alone applications, the Tcl scripts that form the library +have to be integrated into the application as well. One tool supporting that is +SAM (stand-alone modules), which is part of the Tix distribution +(http://tix.sourceforge.net/). + +Build Tix with SAM enabled, perform the appropriate call to +:cfunc:`Tclsam_init`, etc. inside Python's +:file:`Modules/tkappinit.c`, and link with libtclsam and libtksam (you +might include the Tix libraries as well). + + +Can I have Tk events handled while waiting for I/O? +--------------------------------------------------- + +Yes, and you don't even need threads! But you'll have to restructure your I/O +code a bit. Tk has the equivalent of Xt's :cfunc:`XtAddInput()` call, which allows you +to register a callback function which will be called from the Tk mainloop when +I/O is possible on a file descriptor. Here's what you need:: + + from Tkinter import tkinter + tkinter.createfilehandler(file, mask, callback) + +The file may be a Python file or socket object (actually, anything with a +fileno() method), or an integer file descriptor. The mask is one of the +constants tkinter.READABLE or tkinter.WRITABLE. The callback is called as +follows:: + + callback(file, mask) + +You must unregister the callback when you're done, using :: + + tkinter.deletefilehandler(file) + +Note: since you don't know *how many bytes* are available for reading, you can't +use the Python file object's read or readline methods, since these will insist +on reading a predefined number of bytes. For sockets, the :meth:`recv` or +:meth:`recvfrom` methods will work fine; for other files, use +``os.read(file.fileno(), maxbytecount)``. + + +I can't get key bindings to work in Tkinter: why? +------------------------------------------------- + +An often-heard complaint is that event handlers bound to events with the +:meth:`bind` method don't get handled even when the appropriate key is pressed. + +The most common cause is that the widget to which the binding applies doesn't +have "keyboard focus". Check out the Tk documentation for the focus command. +Usually a widget is given the keyboard focus by clicking in it (but not for +labels; see the takefocus option). + + + ======================================= --- /dev/null +++ /faq/index.rst Sat Mar 19 08:59:41 2011 @@ -0,0 +1,18 @@ +################################### + Python Frequently Asked Questions +################################### + +:Release: |version| +:Date: |today| + +.. toctree:: + :maxdepth: 1 + + general.rst + programming.rst + design.rst + library.rst + extending.rst + windows.rst + gui.rst + installed.rst ======================================= --- /dev/null +++ /faq/installed.rst Sat Mar 19 08:59:41 2011 @@ -0,0 +1,53 @@ +============================================= +"Why is Python Installed on my Computer?" FAQ +============================================= + +What is Python? +--------------- + +Python is a programming language. It's used for many different applications. +It's used in some high schools and colleges as an introductory programming +language because Python is easy to learn, but it's also used by professional +software developers at places such as Google, NASA, and Lucasfilm Ltd. + +If you wish to learn more about Python, start with the `Beginner's Guide to +Python <http://wiki.python.org/moin/BeginnersGuide>`_. + + +Why is Python installed on my machine? +-------------------------------------- + +If you find Python installed on your system but don't remember installing it, +there are several possible ways it could have gotten there. + +* Perhaps another user on the computer wanted to learn programming and installed + it; you'll have to figure out who's been using the machine and might have + installed it. +* A third-party application installed on the machine might have been written in + Python and included a Python installation. For a home computer, the most + common such application is `PySol <http://pysolfc.sourceforge.net/>`_, a + solitaire game that includes over 1000 different games and variations. +* Some Windows machines also have Python installed. At this writing we're aware + of computers from Hewlett-Packard and Compaq that include Python. Apparently + some of HP/Compaq's administrative tools are written in Python. +* All Apple computers running Mac OS X have Python installed; it's included in + the base installation. + + +Can I delete Python? +-------------------- + +That depends on where Python came from. + +If someone installed it deliberately, you can remove it without hurting +anything. On Windows, use the Add/Remove Programs icon in the Control Panel. + +If Python was installed by a third-party application, you can also remove it, +but that application will no longer work. You should use that application's +uninstaller rather than removing Python directly. + +If Python came with your operating system, removing it is not recommended. If +you remove it, whatever tools were written in Python will no longer run, and +some of them might be important to you. Reinstalling the whole system would +then be required to fix things again. + ======================================= --- /dev/null +++ /faq/library.rst Sat Mar 19 08:59:41 2011 @@ -0,0 +1,877 @@ +:tocdepth: 2 + +========================= +Library and Extension FAQ +========================= + +.. contents:: + +General Library Questions +========================= + +How do I find a module or application to perform task X? +-------------------------------------------------------- + +Check :ref:`the Library Reference <library-index>` to see if there's a relevant +standard library module. (Eventually you'll learn what's in the standard +library and will able to skip this step.) + +For third-party packages, search the `Python Package Index +<http://pypi.python.org/pypi>`_ or try `Google <http://www.google.com>`_ or +another Web search engine. Searching for "Python" plus a keyword or two for +your topic of interest will usually find something helpful. + + +Where is the math.py (socket.py, regex.py, etc.) source file? +------------------------------------------------------------- + +If you can't find a source file for a module it may be a built-in or +dynamically loaded module implemented in C, C++ or other compiled language. +In this case you may not have the source file or it may be something like +mathmodule.c, somewhere in a C source directory (not on the Python Path). + +There are (at least) three kinds of modules in Python: + +1) modules written in Python (.py); +2) modules written in C and dynamically loaded (.dll, .pyd, .so, .sl, etc); +3) modules written in C and linked with the interpreter; to get a list of these, + type:: + + import sys + print sys.builtin_module_names + + +How do I make a Python script executable on Unix? +------------------------------------------------- + +You need to do two things: the script file's mode must be executable and the +first line must begin with ``#!`` followed by the path of the Python +interpreter. + +The first is done by executing ``chmod +x scriptfile`` or perhaps ``chmod 755 +scriptfile``. + +The second can be done in a number of ways. The most straightforward way is to +write :: + + #!/usr/local/bin/python + +as the very first line of your file, using the pathname for where the Python +interpreter is installed on your platform. + +If you would like the script to be independent of where the Python interpreter +lives, you can use the "env" program. Almost all Unix variants support the +following, assuming the Python interpreter is in a directory on the user's +$PATH:: + + #!/usr/bin/env python + +*Don't* do this for CGI scripts. The $PATH variable for CGI scripts is often +very minimal, so you need to use the actual absolute pathname of the +interpreter. + +Occasionally, a user's environment is so full that the /usr/bin/env program +fails; or there's no env program at all. In that case, you can try the +following hack (due to Alex Rezinsky):: + + #! /bin/sh + """:" + exec python $0 ${1+"$@"} + """ + +The minor disadvantage is that this defines the script's __doc__ string. +However, you can fix that by adding :: + + __doc__ = """...Whatever...""" + + + +Is there a curses/termcap package for Python? +--------------------------------------------- + +.. XXX curses *is* built by default, isn't it? + +For Unix variants: The standard Python source distribution comes with a curses +module in the ``Modules/`` subdirectory, though it's not compiled by default +(note that this is not available in the Windows distribution -- there is no +curses module for Windows). + +The curses module supports basic curses features as well as many additional +functions from ncurses and SYSV curses such as colour, alternative character set +support, pads, and mouse support. This means the module isn't compatible with +operating systems that only have BSD curses, but there don't seem to be any +currently maintained OSes that fall into this category. + +For Windows: use `the consolelib module +<http://effbot.org/zone/console-index.htm>`_. + + +Is there an equivalent to C's onexit() in Python? +------------------------------------------------- + +The :mod:`atexit` module provides a register function that is similar to C's +onexit. + + +Why don't my signal handlers work? +---------------------------------- + +The most common problem is that the signal handler is declared with the wrong +argument list. It is called as :: + + handler(signum, frame) + +so it should be declared with two arguments:: + + def handler(signum, frame): + ... + + +Common tasks +============ + +How do I test a Python program or component? +-------------------------------------------- + +Python comes with two testing frameworks. The :mod:`doctest` module finds +examples in the docstrings for a module and runs them, comparing the output with +the expected output given in the docstring. + +The :mod:`unittest` module is a fancier testing framework modelled on Java and +Smalltalk testing frameworks. + +For testing, it helps to write the program so that it may be easily tested by +using good modular design. Your program should have almost all functionality +encapsulated in either functions or class methods -- and this sometimes has the +surprising and delightful effect of making the program run faster (because local +variable accesses are faster than global accesses). Furthermore the program +should avoid depending on mutating global variables, since this makes testing +much more difficult to do. + +The "global main logic" of your program may be as simple as :: + + if __name__ == "__main__": + main_logic() + +at the bottom of the main module of your program. + +Once your program is organized as a tractable collection of functions and class +behaviours you should write test functions that exercise the behaviours. A test +suite can be associated with each module which automates a sequence of tests. +This sounds like a lot of work, but since Python is so terse and flexible it's +surprisingly easy. You can make coding much more pleasant and fun by writing +your test functions in parallel with the "production code", since this makes it +easy to find bugs and even design flaws earlier. + +"Support modules" that are not intended to be the main module of a program may +include a self-test of the module. :: + + if __name__ == "__main__": + self_test() + +Even programs that interact with complex external interfaces may be tested when +the external interfaces are unavailable by using "fake" interfaces implemented +in Python. + + +How do I create documentation from doc strings? +----------------------------------------------- + +The :mod:`pydoc` module can create HTML from the doc strings in your Python +source code. An alternative for creating API documentation purely from +docstrings is `epydoc <http://epydoc.sf.net/>`_. `Sphinx +<http://sphinx.pocoo.org>`_ can also include docstring content. + + +How do I get a single keypress at a time? +----------------------------------------- + +For Unix variants: There are several solutions. It's straightforward to do this +using curses, but curses is a fairly large module to learn. Here's a solution +without curses:: + + import termios, fcntl, sys, os + fd = sys.stdin.fileno() + + oldterm = termios.tcgetattr(fd) + newattr = termios.tcgetattr(fd) + newattr[3] = newattr[3] & ~termios.ICANON & ~termios.ECHO + termios.tcsetattr(fd, termios.TCSANOW, newattr) + + oldflags = fcntl.fcntl(fd, fcntl.F_GETFL) + fcntl.fcntl(fd, fcntl.F_SETFL, oldflags | os.O_NONBLOCK) + + try: + while 1: + try: + c = sys.stdin.read(1) + print "Got character", repr(c) + except IOError: pass + finally: + termios.tcsetattr(fd, termios.TCSAFLUSH, oldterm) + fcntl.fcntl(fd, fcntl.F_SETFL, oldflags) + +You need the :mod:`termios` and the :mod:`fcntl` module for any of this to work, +and I've only tried it on Linux, though it should work elsewhere. In this code, +characters are read and printed one at a time. + +:func:`termios.tcsetattr` turns off stdin's echoing and disables canonical mode. +:func:`fcntl.fnctl` is used to obtain stdin's file descriptor flags and modify +them for non-blocking mode. Since reading stdin when it is empty results in an +:exc:`IOError`, this error is caught and ignored. + + +Threads +======= + +How do I program using threads? +------------------------------- + +.. XXX it's _thread in py3k + +Be sure to use the :mod:`threading` module and not the :mod:`thread` module. +The :mod:`threading` module builds convenient abstractions on top of the +low-level primitives provided by the :mod:`thread` module. + +Aahz has a set of slides from his threading tutorial that are helpful; see +http://www.pythoncraft.com/OSCON2001/. + + +None of my threads seem to run: why? +------------------------------------ + +As soon as the main thread exits, all threads are killed. Your main thread is +running too quickly, giving the threads no time to do any work. + +A simple fix is to add a sleep to the end of the program that's long enough for +all the threads to finish:: + + import threading, time + + def thread_task(name, n): + for i in range(n): print name, i + + for i in range(10): + T = threading.Thread(target=thread_task, args=(str(i), i)) + T.start() + + time.sleep(10) # <----------------------------! + +But now (on many platforms) the threads don't run in parallel, but appear to run +sequentially, one at a time! The reason is that the OS thread scheduler doesn't +start a new thread until the previous thread is blocked. + +A simple fix is to add a tiny sleep to the start of the run function:: + + def thread_task(name, n): + time.sleep(0.001) # <---------------------! + for i in range(n): print name, i + + for i in range(10): + T = threading.Thread(target=thread_task, args=(str(i), i)) + T.start() + + time.sleep(10) + +Instead of trying to guess how long a :func:`time.sleep` delay will be enough, +it's better to use some kind of semaphore mechanism. One idea is to use the +:mod:`Queue` module to create a queue object, let each thread append a token to +the queue when it finishes, and let the main thread read as many tokens from the +queue as there are threads. + + +How do I parcel out work among a bunch of worker threads? +--------------------------------------------------------- + +Use the :mod:`Queue` module to create a queue containing a list of jobs. The +:class:`~Queue.Queue` class maintains a list of objects with ``.put(obj)`` to +add an item to the queue and ``.get()`` to return an item. The class will take +care of the locking necessary to ensure that each job is handed out exactly +once. + +Here's a trivial example:: + + import threading, Queue, time + + # The worker thread gets jobs off the queue. When the queue is empty, it + # assumes there will be no more work and exits. + # (Realistically workers will run until terminated.) + def worker (): + print 'Running worker' + time.sleep(0.1) + while True: + try: + arg = q.get(block=False) + except Queue.Empty: + print 'Worker', threading.currentThread(), + print 'queue empty' + break + else: + print 'Worker', threading.currentThread(), + print 'running with argument', arg + time.sleep(0.5) + + # Create queue + q = Queue.Queue() + + # Start a pool of 5 workers + for i in range(5): + t = threading.Thread(target=worker, name='worker %i' % (i+1)) + t.start() + + # Begin adding work to the queue + for i in range(50): + q.put(i) + + # Give threads time to run + print 'Main thread sleeping' + time.sleep(5) + +When run, this will produce the following output: + + Running worker + Running worker + Running worker + Running worker + Running worker + Main thread sleeping + Worker <Thread(worker 1, started)> running with argument 0 + Worker <Thread(worker 2, started)> running with argument 1 + Worker <Thread(worker 3, started)> running with argument 2 + Worker <Thread(worker 4, started)> running with argument 3 + Worker <Thread(worker 5, started)> running with argument 4 + Worker <Thread(worker 1, started)> running with argument 5 + ... + +Consult the module's documentation for more details; the ``Queue`` class +provides a featureful interface. + + +What kinds of global value mutation are thread-safe? +---------------------------------------------------- + +A global interpreter lock (GIL) is used internally to ensure that only one +thread runs in the Python VM at a time. In general, Python offers to switch +among threads only between bytecode instructions; how frequently it switches can +be set via :func:`sys.setcheckinterval`. Each bytecode instruction and +therefore all the C implementation code reached from each instruction is +therefore atomic from the point of view of a Python program. + +In theory, this means an exact accounting requires an exact understanding of the +PVM bytecode implementation. In practice, it means that operations on shared +variables of built-in data types (ints, lists, dicts, etc) that "look atomic" +really are. + +For example, the following operations are all atomic (L, L1, L2 are lists, D, +D1, D2 are dicts, x, y are objects, i, j are ints):: + + L.append(x) + L1.extend(L2) + x = L[i] + x = L.pop() + L1[i:j] = L2 + L.sort() + x = y + x.field = y + D[x] = y + D1.update(D2) + D.keys() + +These aren't:: + + i = i+1 + L.append(L[-1]) + L[i] = L[j] + D[x] = D[x] + 1 + +Operations that replace other objects may invoke those other objects' +:meth:`__del__` method when their reference count reaches zero, and that can +affect things. This is especially true for the mass updates to dictionaries and +lists. When in doubt, use a mutex! + + +Can't we get rid of the Global Interpreter Lock? +------------------------------------------------ + +.. XXX mention multiprocessing +.. XXX link to dbeazley's talk about GIL? + +The Global Interpreter Lock (GIL) is often seen as a hindrance to Python's +deployment on high-end multiprocessor server machines, because a multi-threaded +Python program effectively only uses one CPU, due to the insistence that +(almost) all Python code can only run while the GIL is held. + +Back in the days of Python 1.5, Greg Stein actually implemented a comprehensive +patch set (the "free threading" patches) that removed the GIL and replaced it +with fine-grained locking. Unfortunately, even on Windows (where locks are very +efficient) this ran ordinary Python code about twice as slow as the interpreter +using the GIL. On Linux the performance loss was even worse because pthread +locks aren't as efficient. + +Since then, the idea of getting rid of the GIL has occasionally come up but +nobody has found a way to deal with the expected slowdown, and users who don't +use threads would not be happy if their code ran at half at the speed. Greg's +free threading patch set has not been kept up-to-date for later Python versions. + +This doesn't mean that you can't make good use of Python on multi-CPU machines! +You just have to be creative with dividing the work up between multiple +*processes* rather than multiple *threads*. Judicious use of C extensions will +also help; if you use a C extension to perform a time-consuming task, the +extension can release the GIL while the thread of execution is in the C code and +allow other threads to get some work done. + +It has been suggested that the GIL should be a per-interpreter-state lock rather +than truly global; interpreters then wouldn't be able to share objects. +Unfortunately, this isn't likely to happen either. It would be a tremendous +amount of work, because many object implementations currently have global state. +For example, small integers and short strings are cached; these caches would +have to be moved to the interpreter state. Other object types have their own +free list; these free lists would have to be moved to the interpreter state. +And so on. + +And I doubt that it can even be done in finite time, because the same problem +exists for 3rd party extensions. It is likely that 3rd party extensions are +being written at a faster rate than you can convert them to store all their +global state in the interpreter state. + +And finally, once you have multiple interpreters not sharing any state, what +have you gained over running each interpreter in a separate process? + + +Input and Output +================ + +How do I delete a file? (And other file questions...) +----------------------------------------------------- + +Use ``os.remove(filename)`` or ``os.unlink(filename)``; for documentation, see +the :mod:`os` module. The two functions are identical; :func:`unlink` is simply +the name of the Unix system call for this function. + +To remove a directory, use :func:`os.rmdir`; use :func:`os.mkdir` to create one. +``os.makedirs(path)`` will create any intermediate directories in ``path`` that +don't exist. ``os.removedirs(path)`` will remove intermediate directories as +long as they're empty; if you want to delete an entire directory tree and its +contents, use :func:`shutil.rmtree`. + +To rename a file, use ``os.rename(old_path, new_path)``. + +To truncate a file, open it using ``f = open(filename, "r+")``, and use +``f.truncate(offset)``; offset defaults to the current seek position. There's +also ```os.ftruncate(fd, offset)`` for files opened with :func:`os.open`, where +``fd`` is the file descriptor (a small integer). + +The :mod:`shutil` module also contains a number of functions to work on files +including :func:`~shutil.copyfile`, :func:`~shutil.copytree`, and +:func:`~shutil.rmtree`. + + +How do I copy a file? +--------------------- + +The :mod:`shutil` module contains a :func:`~shutil.copyfile` function. Note +that on MacOS 9 it doesn't copy the resource fork and Finder info. + + +How do I read (or write) binary data? +------------------------------------- + +To read or write complex binary data formats, it's best to use the :mod:`struct` +module. It allows you to take a string containing binary data (usually numbers) +and convert it to Python objects; and vice versa. + +For example, the following code reads two 2-byte integers and one 4-byte integer +in big-endian format from a file:: + + import struct + + f = open(filename, "rb") # Open in binary mode for portability + s = f.read(8) + x, y, z = struct.unpack(">hhl", s) + +The '>' in the format string forces big-endian data; the letter 'h' reads one +"short integer" (2 bytes), and 'l' reads one "long integer" (4 bytes) from the +string. + +For data that is more regular (e.g. a homogeneous list of ints or thefloats), +you can also use the :mod:`array` module. + + +I can't seem to use os.read() on a pipe created with os.popen(); why? +--------------------------------------------------------------------- + +:func:`os.read` is a low-level function which takes a file descriptor, a small +integer representing the opened file. :func:`os.popen` creates a high-level +file object, the same type returned by the built-in :func:`open` function. +Thus, to read n bytes from a pipe p created with :func:`os.popen`, you need to +use ``p.read(n)``. + + +How do I run a subprocess with pipes connected to both input and output? +------------------------------------------------------------------------ + +.. XXX update to use subprocess + +Use the :mod:`popen2` module. For example:: + + import popen2 + fromchild, tochild = popen2.popen2("command") + tochild.write("input\n") + tochild.flush() + output = fromchild.readline() + +Warning: in general it is unwise to do this because you can easily cause a +deadlock where your process is blocked waiting for output from the child while +the child is blocked waiting for input from you. This can be caused because the +parent expects the child to output more text than it does, or it can be caused +by data being stuck in stdio buffers due to lack of flushing. The Python parent +can of course explicitly flush the data it sends to the child before it reads +any output, but if the child is a naive C program it may have been written to +never explicitly flush its output, even if it is interactive, since flushing is +normally automatic. + +Note that a deadlock is also possible if you use :func:`popen3` to read stdout +and stderr. If one of the two is too large for the internal buffer (increasing +the buffer size does not help) and you ``read()`` the other one first, there is +a deadlock, too. + +Note on a bug in popen2: unless your program calls ``wait()`` or ``waitpid()``, +finished child processes are never removed, and eventually calls to popen2 will +fail because of a limit on the number of child processes. Calling +:func:`os.waitpid` with the :data:`os.WNOHANG` option can prevent this; a good +place to insert such a call would be before calling ``popen2`` again. + +In many cases, all you really need is to run some data through a command and get +the result back. Unless the amount of data is very large, the easiest way to do +this is to write it to a temporary file and run the command with that temporary +file as input. The standard module :mod:`tempfile` exports a ``mktemp()`` +function to generate unique temporary file names. :: + + import tempfile + import os + + class Popen3: + """ + This is a deadlock-safe version of popen that returns + an object with errorlevel, out (a string) and err (a string). + (capturestderr may not work under windows.) + Example: print Popen3('grep spam','\n\nhere spam\n\n').out + """ + def __init__(self,command,input=None,capturestderr=None): + outfile=tempfile.mktemp() + command="( %s ) > %s" % (command,outfile) + if input: + infile=tempfile.mktemp() + open(infile,"w").write(input) + command=command+" <"+infile + if capturestderr: + errfile=tempfile.mktemp() + command=command+" 2>"+errfile + self.errorlevel=os.system(command) >> 8 + self.out=open(outfile,"r").read() + os.remove(outfile) + if input: + os.remove(infile) + if capturestderr: + self.err=open(errfile,"r").read() + os.remove(errfile) + +Note that many interactive programs (e.g. vi) don't work well with pipes +substituted for standard input and output. You will have to use pseudo ttys +("ptys") instead of pipes. Or you can use a Python interface to Don Libes' +"expect" library. A Python extension that interfaces to expect is called "expy" +and available from http://expectpy.sourceforge.net. A pure Python solution that +works like expect is `pexpect <http://pypi.python.org/pypi/pexpect/>`_. + + +How do I access the serial (RS232) port? +---------------------------------------- + +For Win32, POSIX (Linux, BSD, etc.), Jython: + + http://pyserial.sourceforge.net + +For Unix, see a Usenet post by Mitch Chapman: + + http://groups.google.com/groups?selm=****@ohioe***** + + +Why doesn't closing sys.stdout (stdin, stderr) really close it? +--------------------------------------------------------------- + +Python file objects are a high-level layer of abstraction on top of C streams, +which in turn are a medium-level layer of abstraction on top of (among other +things) low-level C file descriptors. + +For most file objects you create in Python via the built-in ``file`` +constructor, ``f.close()`` marks the Python file object as being closed from +Python's point of view, and also arranges to close the underlying C stream. +This also happens automatically in ``f``'s destructor, when ``f`` becomes +garbage. + +But stdin, stdout and stderr are treated specially by Python, because of the +special status also given to them by C. Running ``sys.stdout.close()`` marks +the Python-level file object as being closed, but does *not* close the +associated C stream. + +To close the underlying C stream for one of these three, you should first be +sure that's what you really want to do (e.g., you may confuse extension modules +trying to do I/O). If it is, use os.close:: + + os.close(0) # close C's stdin stream + os.close(1) # close C's stdout stream + os.close(2) # close C's stderr stream + + +Network/Internet Programming +============================ + +What WWW tools are there for Python? +------------------------------------ + +See the chapters titled :ref:`internet` and :ref:`netdata` in the Library +Reference Manual. Python has many modules that will help you build server-side +and client-side web systems. + +.. XXX check if wiki page is still up to date + +A summary of available frameworks is maintained by Paul Boddie at +http://wiki.python.org/moin/WebProgramming . + +Cameron Laird maintains a useful set of pages about Python web technologies at +http://phaseit.net/claird/comp.lang.python/web_python. + + +How can I mimic CGI form submission (METHOD=POST)? +-------------------------------------------------- + +I would like to retrieve web pages that are the result of POSTing a form. Is +there existing code that would let me do this easily? + +Yes. Here's a simple example that uses httplib:: + + #!/usr/local/bin/python + + import httplib, sys, time + + ### build the query string + qs = "First=Josephine&MI=Q&Last=Public" + + ### connect and send the server a path + httpobj = httplib.HTTP('www.some-server.out-there', 80) + httpobj.putrequest('POST', '/cgi-bin/some-cgi-script') + ### now generate the rest of the HTTP headers... + httpobj.putheader('Accept', '*/*') + httpobj.putheader('Connection', 'Keep-Alive') + httpobj.putheader('Content-type', 'application/x-www-form-urlencoded') + httpobj.putheader('Content-length', '%d' % len(qs)) + httpobj.endheaders() + httpobj.send(qs) + ### find out what the server said in response... + reply, msg, hdrs = httpobj.getreply() + if reply != 200: + sys.stdout.write(httpobj.getfile().read()) + +Note that in general for URL-encoded POST operations, query strings must be +quoted by using :func:`urllib.quote`. For example to send name="Guy Steele, +Jr.":: + + >>> from urllib import quote + >>> x = quote("Guy Steele, Jr.") + >>> x + 'Guy%20Steele,%20Jr.' + >>> query_string = "name="+x + >>> query_string + 'name=Guy%20Steele,%20Jr.' + + +What module should I use to help with generating HTML? +------------------------------------------------------ + +.. XXX add modern template languages + +There are many different modules available: + +* HTMLgen is a class library of objects corresponding to all the HTML 3.2 markup + tags. It's used when you are writing in Python and wish to synthesize HTML + pages for generating a web or for CGI forms, etc. + +* DocumentTemplate and Zope Page Templates are two different systems that are + part of Zope. + +* Quixote's PTL uses Python syntax to assemble strings of text. + +Consult the `Web Programming wiki pages +<http://wiki.python.org/moin/WebProgramming>`_ for more links. + + +How do I send mail from a Python script? +---------------------------------------- + +Use the standard library module :mod:`smtplib`. + +Here's a very simple interactive mail sender that uses it. This method will +work on any host that supports an SMTP listener. :: + + import sys, smtplib + + fromaddr = raw_input("From: ") + toaddrs = raw_input("To: ").split(',') + print "Enter message, end with ^D:" + msg = '' + while True: + line = sys.stdin.readline() + if not line: + break + msg += line + + # The actual mail send + server = smtplib.SMTP('localhost') + server.sendmail(fromaddr, toaddrs, msg) + server.quit() + +A Unix-only alternative uses sendmail. The location of the sendmail program +varies between systems; sometimes it is ``/usr/lib/sendmail``, sometime +``/usr/sbin/sendmail``. The sendmail manual page will help you out. Here's +some sample code:: + + SENDMAIL = "/usr/sbin/sendmail" # sendmail location + import os + p = os.popen("%s -t -i" % SENDMAIL, "w") + p.write("To: recei****@examp*****\n") + p.write("Subject: test\n") + p.write("\n") # blank line separating headers from body + p.write("Some text\n") + p.write("some more text\n") + sts = p.close() + if sts != 0: + print "Sendmail exit status", sts + + +How do I avoid blocking in the connect() method of a socket? +------------------------------------------------------------ + +The select module is commonly used to help with asynchronous I/O on sockets. + +To prevent the TCP connect from blocking, you can set the socket to non-blocking +mode. Then when you do the ``connect()``, you will either connect immediately +(unlikely) or get an exception that contains the error number as ``.errno``. +``errno.EINPROGRESS`` indicates that the connection is in progress, but hasn't +finished yet. Different OSes will return different values, so you're going to +have to check what's returned on your system. + +You can use the ``connect_ex()`` method to avoid creating an exception. It will +just return the errno value. To poll, you can call ``connect_ex()`` again later +-- 0 or ``errno.EISCONN`` indicate that you're connected -- or you can pass this +socket to select to check if it's writable. + + +Databases +========= + +Are there any interfaces to database packages in Python? +-------------------------------------------------------- + +Yes. + +.. XXX remove bsddb in py3k, fix other module names + +Python 2.3 includes the :mod:`bsddb` package which provides an interface to the +BerkeleyDB library. Interfaces to disk-based hashes such as :mod:`DBM <dbm>` +and :mod:`GDBM <gdbm>` are also included with standard Python. + +Support for most relational databases is available. See the +`DatabaseProgramming wiki page +<http://wiki.python.org/moin/DatabaseProgramming>`_ for details. + + +How do you implement persistent objects in Python? +-------------------------------------------------- + +The :mod:`pickle` library module solves this in a very general way (though you +still can't store things like open files, sockets or windows), and the +:mod:`shelve` library module uses pickle and (g)dbm to create persistent +mappings containing arbitrary Python objects. For better performance, you can +use the :mod:`cPickle` module. + +A more awkward way of doing things is to use pickle's little sister, marshal. +The :mod:`marshal` module provides very fast ways to store noncircular basic +Python types to files and strings, and back again. Although marshal does not do +fancy things like store instances or handle shared references properly, it does +run extremely fast. For example loading a half megabyte of data may take less +than a third of a second. This often beats doing something more complex and +general such as using gdbm with pickle/shelve. + + +Why is cPickle so slow? +----------------------- + +.. XXX update this, default protocol is 2/3 + +The default format used by the pickle module is a slow one that results in +readable pickles. Making it the default, but it would break backward +compatibility:: + + largeString = 'z' * (100 * 1024) + myPickle = cPickle.dumps(largeString, protocol=1) + + +If my program crashes with a bsddb (or anydbm) database open, it gets corrupted. How come? +------------------------------------------------------------------------------------------ + +Databases opened for write access with the bsddb module (and often by the anydbm +module, since it will preferentially use bsddb) must explicitly be closed using +the ``.close()`` method of the database. The underlying library caches database +contents which need to be converted to on-disk form and written. + +If you have initialized a new bsddb database but not written anything to it +before the program crashes, you will often wind up with a zero-length file and +encounter an exception the next time the file is opened. + + +I tried to open Berkeley DB file, but bsddb produces bsddb.error: (22, 'Invalid argument'). Help! How can I restore my data? +---------------------------------------------------------------------------------------------------------------------------- + +Don't panic! Your data is probably intact. The most frequent cause for the error +is that you tried to open an earlier Berkeley DB file with a later version of +the Berkeley DB library. + +Many Linux systems now have all three versions of Berkeley DB available. If you +are migrating from version 1 to a newer version use db_dump185 to dump a plain +text version of the database. If you are migrating from version 2 to version 3 +use db2_dump to create a plain text version of the database. In either case, +use db_load to create a new native database for the latest version installed on +your computer. If you have version 3 of Berkeley DB installed, you should be +able to use db2_load to create a native version 2 database. + +You should move away from Berkeley DB version 1 files because the hash file code +contains known bugs that can corrupt your data. + + +Mathematics and Numerics +======================== + +How do I generate random numbers in Python? +------------------------------------------- + +The standard module :mod:`random` implements a random number generator. Usage +is simple:: + + import random + random.random() + +This returns a random floating point number in the range [0, 1). + +There are also many other specialized generators in this module, such as: + +* ``randrange(a, b)`` chooses an integer in the range [a, b). +* ``uniform(a, b)`` chooses a floating point number in the range [a, b). +* ``normalvariate(mean, sdev)`` samples the normal (Gaussian) distribution. + +Some higher-level functions operate on sequences directly, such as: + +* ``choice(S)`` chooses random element from a given sequence +* ``shuffle(L)`` shuffles a list in-place, i.e. permutes it randomly + +There's also a ``Random`` class you can instantiate to create independent +multiple random number generators. ======================================= --- /dev/null +++ /faq/programming.rst Sat Mar 19 08:59:41 2011 @@ -0,0 +1,1776 @@ +:tocdepth: 2 + +=============== +Programming FAQ +=============== + +.. contents:: + +General Questions +================= + +Is there a source code level debugger with breakpoints, single-stepping, etc.? +------------------------------------------------------------------------------ + +Yes. + +The pdb module is a simple but adequate console-mode debugger for Python. It is +part of the standard Python library, and is :mod:`documented in the Library +Reference Manual <pdb>`. You can also write your own debugger by using the code +for pdb as an example. + +The IDLE interactive development environment, which is part of the standard +Python distribution (normally available as Tools/scripts/idle), includes a +graphical debugger. There is documentation for the IDLE debugger at +http://www.python.org/idle/doc/idle2.html#Debugger. + +PythonWin is a Python IDE that includes a GUI debugger based on pdb. The +Pythonwin debugger colors breakpoints and has quite a few cool features such as +debugging non-Pythonwin programs. Pythonwin is available as part of the `Python +for Windows Extensions <http://sourceforge.net/projects/pywin32/>`__ project and +as a part of the ActivePython distribution (see +http://www.activestate.com/Products/ActivePython/index.html). + +`Boa Constructor <http://boa-constructor.sourceforge.net/>`_ is an IDE and GUI +builder that uses wxWidgets. It offers visual frame creation and manipulation, +an object inspector, many views on the source like object browsers, inheritance +hierarchies, doc string generated html documentation, an advanced debugger, +integrated help, and Zope support. + +`Eric <http://www.die-offenbachs.de/eric/index.html>`_ is an IDE built on PyQt +and the Scintilla editing component. + +Pydb is a version of the standard Python debugger pdb, modified for use with DDD +(Data Display Debugger), a popular graphical debugger front end. Pydb can be +found at http://bashdb.sourceforge.net/pydb/ and DDD can be found at +http://www.gnu.org/software/ddd. + +There are a number of commercial Python IDEs that include graphical debuggers. +They include: + +* Wing IDE (http://wingware.com/) +* Komodo IDE (http://www.activestate.com/Products/Komodo) + + +Is there a tool to help find bugs or perform static analysis? +------------------------------------------------------------- + +Yes. + +PyChecker is a static analysis tool that finds bugs in Python source code and +warns about code complexity and style. You can get PyChecker from +http://pychecker.sf.net. + +`Pylint <http://www.logilab.org/projects/pylint>`_ is another tool that checks +if a module satisfies a coding standard, and also makes it possible to write +plug-ins to add a custom feature. In addition to the bug checking that +PyChecker performs, Pylint offers some additional features such as checking line +length, whether variable names are well-formed according to your coding +standard, whether declared interfaces are fully implemented, and more. +http://www.logilab.org/card/pylint_manual provides a full list of Pylint's +features. + + +How can I create a stand-alone binary from a Python script? +----------------------------------------------------------- + +You don't need the ability to compile Python to C code if all you want is a +stand-alone program that users can download and run without having to install +the Python distribution first. There are a number of tools that determine the +set of modules required by a program and bind these modules together with a +Python binary to produce a single executable. + +One is to use the freeze tool, which is included in the Python source tree as +``Tools/freeze``. It converts Python byte code to C arrays; a C compiler you can +embed all your modules into a new program, which is then linked with the +standard Python modules. + +It works by scanning your source recursively for import statements (in both +forms) and looking for the modules in the standard Python path as well as in the +source directory (for built-in modules). It then turns the bytecode for modules +written in Python into C code (array initializers that can be turned into code +objects using the marshal module) and creates a custom-made config file that +only contains those built-in modules which are actually used in the program. It +then compiles the generated C code and links it with the rest of the Python +interpreter to form a self-contained binary which acts exactly like your script. + +Obviously, freeze requires a C compiler. There are several other utilities +which don't. One is Thomas Heller's py2exe (Windows only) at + + http://www.py2exe.org/ + +Another is Christian Tismer's `SQFREEZE <http://starship.python.net/crew/pirx>`_ +which appends the byte code to a specially-prepared Python interpreter that can +find the byte code in the executable. + +Other tools include Fredrik Lundh's `Squeeze +<http://www.pythonware.com/products/python/squeeze>`_ and Anthony Tuininga's +`cx_Freeze <http://starship.python.net/crew/atuining/cx_Freeze/index.html>`_. + + +Are there coding standards or a style guide for Python programs? +---------------------------------------------------------------- + +Yes. The coding style required for standard library modules is documented as +:pep:`8`. + + +My program is too slow. How do I speed it up? +--------------------------------------------- + +That's a tough one, in general. There are many tricks to speed up Python code; +consider rewriting parts in C as a last resort. + +In some cases it's possible to automatically translate Python to C or x86 +assembly language, meaning that you don't have to modify your code to gain +increased speed. + +.. XXX seems to have overlap with other questions! + +`Pyrex <http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/>`_ can compile a +slightly modified version of Python code into a C extension, and can be used on +many different platforms. + +`Psyco <http://psyco.sourceforge.net>`_ is a just-in-time compiler that +translates Python code into x86 assembly language. If you can use it, Psyco can +provide dramatic speedups for critical functions. + +The rest of this answer will discuss various tricks for squeezing a bit more +speed out of Python code. *Never* apply any optimization tricks unless you know +you need them, after profiling has indicated that a particular function is the +heavily executed hot spot in the code. Optimizations almost always make the +code less clear, and you shouldn't pay the costs of reduced clarity (increased +development time, greater likelihood of bugs) unless the resulting performance +benefit is worth it. + +There is a page on the wiki devoted to `performance tips +<http://wiki.python.org/moin/PythonSpeed/PerformanceTips>`_. + +Guido van Rossum has written up an anecdote related to optimization at +http://www.python.org/doc/essays/list2str.html. + +One thing to notice is that function and (especially) method calls are rather +expensive; if you have designed a purely OO interface with lots of tiny +functions that don't do much more than get or set an instance variable or call +another method, you might consider using a more direct way such as directly +accessing instance variables. Also see the standard module :mod:`profile` which +makes it possible to find out where your program is spending most of its time +(if you have some patience -- the profiling itself can slow your program down by +an order of magnitude). + +Remember that many standard optimization heuristics you may know from other +programming experience may well apply to Python. For example it may be faster +to send output to output devices using larger writes rather than smaller ones in +order to reduce the overhead of kernel system calls. Thus CGI scripts that +write all output in "one shot" may be faster than those that write lots of small +pieces of output. + +Also, be sure to use Python's core features where appropriate. For example, +slicing allows programs to chop up lists and other sequence objects in a single +tick of the interpreter's mainloop using highly optimized C implementations. +Thus to get the same effect as:: + + L2 = [] + for i in range[3]: + L2.append(L1[i]) + +it is much shorter and far faster to use :: + + L2 = list(L1[:3]) # "list" is redundant if L1 is a list. + +Note that the functionally-oriented built-in functions such as :func:`map`, +:func:`zip`, and friends can be a convenient accelerator for loops that +perform a single task. For example to pair the elements of two lists +together:: + + >>> zip([1, 2, 3], [4, 5, 6]) + [(1, 4), (2, 5), (3, 6)] + +or to compute a number of sines:: + + >>> map(math.sin, (1, 2, 3, 4)) + [0.841470984808, 0.909297426826, 0.14112000806, -0.756802495308] + +The operation completes very quickly in such cases. + +Other examples include the ``join()`` and ``split()`` :ref:`methods +of string objects <string-methods>`. +For example if s1..s7 are large (10K+) strings then +``"".join([s1,s2,s3,s4,s5,s6,s7])`` may be far faster than the more obvious +``s1+s2+s3+s4+s5+s6+s7``, since the "summation" will compute many +subexpressions, whereas ``join()`` does all the copying in one pass. For +manipulating strings, use the ``replace()`` and the ``format()`` :ref:`methods +on string objects <string-methods>`. Use regular expressions only when you're +not dealing with constant string patterns. You may still use :ref:`the old % +operations <string-formatting>` ``string % tuple`` and ``string % dictionary``. + +Be sure to use the :meth:`list.sort` built-in method to do sorting, and see the +`sorting mini-HOWTO <http://wiki.python.org/moin/HowTo/Sorting>`_ for examples +of moderately advanced usage. :meth:`list.sort` beats other techniques for +sorting in all but the most extreme circumstances. + +Another common trick is to "push loops into functions or methods." For example +suppose you have a program that runs slowly and you use the profiler to +determine that a Python function ``ff()`` is being called lots of times. If you +notice that ``ff()``:: + + def ff(x): + ... # do something with x computing result... + return result + +tends to be called in loops like:: + + list = map(ff, oldlist) + +or:: + + for x in sequence: + value = ff(x) + ... # do something with value... + +then you can often eliminate function call overhead by rewriting ``ff()`` to:: + + def ffseq(seq): + resultseq = [] + for x in seq: + ... # do something with x computing result... + resultseq.append(result) + return resultseq + +and rewrite the two examples to ``list = ffseq(oldlist)`` and to:: + + for value in ffseq(sequence): + ... # do something with value... + +Single calls to ``ff(x)`` translate to ``ffseq([x])[0]`` with little penalty. +Of course this technique is not always appropriate and there are other variants +which you can figure out. + +You can gain some performance by explicitly storing the results of a function or +method lookup into a local variable. A loop like:: + + for key in token: + dict[key] = dict.get(key, 0) + 1 + +resolves ``dict.get`` every iteration. If the method isn't going to change, a +slightly faster implementation is:: + + dict_get = dict.get # look up the method once + for key in token: + dict[key] = dict_get(key, 0) + 1 + +Default arguments can be used to determine values once, at compile time instead +of at run time. This can only be done for functions or objects which will not +be changed during program execution, such as replacing :: + + def degree_sin(deg): + return math.sin(deg * math.pi / 180.0) + +with :: + + def degree_sin(deg, factor=math.pi/180.0, sin=math.sin): + return sin(deg * factor) + +Because this trick uses default arguments for terms which should not be changed, +it should only be used when you are not concerned with presenting a possibly +confusing API to your users. + + +Core Language +============= + +Why am I getting an UnboundLocalError when the variable has a value? +-------------------------------------------------------------------- + +It can be a surprise to get the UnboundLocalError in previously working +code when it is modified by adding an assignment statement somewhere in +the body of a function. + +This code: + + >>> x = 10 + >>> def bar(): + ... print x + >>> bar() + 10 + +works, but this code: + + >>> x = 10 + >>> def foo(): + ... print x + ... x += 1 + +results in an UnboundLocalError: + + >>> foo() + Traceback (most recent call last): + ... + UnboundLocalError: local variable 'x' referenced before assignment + +This is because when you make an assignment to a variable in a scope, that +variable becomes local to that scope and shadows any similarly named variable +in the outer scope. Since the last statement in foo assigns a new value to +``x``, the compiler recognizes it as a local variable. Consequently when the +earlier ``print x`` attempts to print the uninitialized local variable and +an error results. + +In the example above you can access the outer scope variable by declaring it +global: + + >>> x = 10 + >>> def foobar(): + ... global x + ... print x + ... x += 1 + >>> foobar() + 10 + +This explicit declaration is required in order to remind you that (unlike the +superficially analogous situation with class and instance variables) you are +actually modifying the value of the variable in the outer scope: + + >>> print x + 11 + + +What are the rules for local and global variables in Python? +------------------------------------------------------------ + +In Python, variables that are only referenced inside a function are implicitly +global. If a variable is assigned a new value anywhere within the function's +body, it's assumed to be a local. If a variable is ever assigned a new value +inside the function, the variable is implicitly local, and you need to +explicitly declare it as 'global'. + +Though a bit surprising at first, a moment's consideration explains this. On +one hand, requiring :keyword:`global` for assigned variables provides a bar +against unintended side-effects. On the other hand, if ``global`` was required +for all global references, you'd be using ``global`` all the time. You'd have +to declare as global every reference to a built-in function or to a component of +an imported module. This clutter would defeat the usefulness of the ``global`` +declaration for identifying side-effects. + + +How do I share global variables across modules? +------------------------------------------------ + +The canonical way to share information across modules within a single program is +to create a special module (often called config or cfg). Just import the config +module in all modules of your application; the module then becomes available as +a global name. Because there is only one instance of each module, any changes +made to the module object get reflected everywhere. For example: + +config.py:: + + x = 0 # Default value of the 'x' configuration setting + +mod.py:: + + import config + config.x = 1 + +main.py:: + + import config + import mod + print config.x + +Note that using a module is also the basis for implementing the Singleton design +pattern, for the same reason. + + +What are the "best practices" for using import in a module? +----------------------------------------------------------- + +In general, don't use ``from modulename import *``. Doing so clutters the +importer's namespace. Some people avoid this idiom even with the few modules +that were designed to be imported in this manner. Modules designed in this +manner include :mod:`Tkinter`, and :mod:`threading`. + +Import modules at the top of a file. Doing so makes it clear what other modules +your code requires and avoids questions of whether the module name is in scope. +Using one import per line makes it easy to add and delete module imports, but +using multiple imports per line uses less screen space. + +It's good practice if you import modules in the following order: + +1. standard library modules -- e.g. ``sys``, ``os``, ``getopt``, ``re`` +2. third-party library modules (anything installed in Python's site-packages + directory) -- e.g. mx.DateTime, ZODB, PIL.Image, etc. +3. locally-developed modules + +Never use relative package imports. If you're writing code that's in the +``package.sub.m1`` module and want to import ``package.sub.m2``, do not just +write ``import m2``, even though it's legal. Write ``from package.sub import +m2`` instead. Relative imports can lead to a module being initialized twice, +leading to confusing bugs. See :pep:`328` for details. + +It is sometimes necessary to move imports to a function or class to avoid +problems with circular imports. Gordon McMillan says: + + Circular imports are fine where both modules use the "import <module>" form + of import. They fail when the 2nd module wants to grab a name out of the + first ("from module import name") and the import is at the top level. That's + because names in the 1st are not yet available, because the first module is + busy importing the 2nd. + +In this case, if the second module is only used in one function, then the import +can easily be moved into that function. By the time the import is called, the +first module will have finished initializing, and the second module can do its +import. + +It may also be necessary to move imports out of the top level of code if some of +the modules are platform-specific. In that case, it may not even be possible to +import all of the modules at the top of the file. In this case, importing the +correct modules in the corresponding platform-specific code is a good option. + +Only move imports into a local scope, such as inside a function definition, if +it's necessary to solve a problem such as avoiding a circular import or are +trying to reduce the initialization time of a module. This technique is +especially helpful if many of the imports are unnecessary depending on how the +program executes. You may also want to move imports into a function if the +modules are only ever used in that function. Note that loading a module the +first time may be expensive because of the one time initialization of the +module, but loading a module multiple times is virtually free, costing only a +couple of dictionary lookups. Even if the module name has gone out of scope, +the module is probably available in :data:`sys.modules`. + +If only instances of a specific class use a module, then it is reasonable to +import the module in the class's ``__init__`` method and then assign the module +to an instance variable so that the module is always available (via that +instance variable) during the life of the object. Note that to delay an import +until the class is instantiated, the import must be inside a method. Putting +the import inside the class but outside of any method still causes the import to +occur when the module is initialized. + + +How can I pass optional or keyword parameters from one function to another? +--------------------------------------------------------------------------- + +Collect the arguments using the ``*`` and ``**`` specifiers in the function's +parameter list; this gives you the positional arguments as a tuple and the +keyword arguments as a dictionary. You can then pass these arguments when +calling another function by using ``*`` and ``**``:: + + def f(x, *args, **kwargs): + ... + kwargs['width'] = '14.3c' + ... + g(x, *args, **kwargs) + +In the unlikely case that you care about Python versions older than 2.0, use +:func:`apply`:: + + def f(x, *args, **kwargs): + ... + kwargs['width'] = '14.3c' + ... + apply(g, (x,)+args, kwargs) + + +How do I write a function with output parameters (call by reference)? +--------------------------------------------------------------------- + +Remember that arguments are passed by assignment in Python. Since assignment +just creates references to objects, there's no alias between an argument name in +the caller and callee, and so no call-by-reference per se. You can achieve the +desired effect in a number of ways. + +1) By returning a tuple of the results:: + + def func2(a, b): + a = 'new-value' # a and b are local names + b = b + 1 # assigned to new objects + return a, b # return new values + + x, y = 'old-value', 99 + x, y = func2(x, y) + print x, y # output: new-value 100 + + This is almost always the clearest solution. + +2) By using global variables. This isn't thread-safe, and is not recommended. + +3) By passing a mutable (changeable in-place) object:: + + def func1(a): + a[0] = 'new-value' # 'a' references a mutable list + a[1] = a[1] + 1 # changes a shared object + + args = ['old-value', 99] + func1(args) + print args[0], args[1] # output: new-value 100 + +4) By passing in a dictionary that gets mutated:: + + def func3(args): + args['a'] = 'new-value' # args is a mutable dictionary + args['b'] = args['b'] + 1 # change it in-place + + args = {'a':' old-value', 'b': 99} + func3(args) + print args['a'], args['b'] + +5) Or bundle up values in a class instance:: + + class callByRef: + def __init__(self, **args): + for (key, value) in args.items(): + setattr(self, key, value) + + def func4(args): + args.a = 'new-value' # args is a mutable callByRef + args.b = args.b + 1 # change object in-place + + args = callByRef(a='old-value', b=99) + func4(args) + print args.a, args.b + + + There's almost never a good reason to get this complicated. + +Your best choice is to return a tuple containing the multiple results. + + +How do you make a higher order function in Python? +-------------------------------------------------- + +You have two choices: you can use nested scopes or you can use callable objects. +For example, suppose you wanted to define ``linear(a,b)`` which returns a +function ``f(x)`` that computes the value ``a*x+b``. Using nested scopes:: + + def linear(a, b): + def result(x): + return a * x + b + return result + +Or using a callable object:: + + class linear: + + def __init__(self, a, b): + self.a, self.b = a, b + + def __call__(self, x): + return self.a * x + self.b + +In both cases, :: + + taxes = linear(0.3, 2) + +gives a callable object where ``taxes(10e6) == 0.3 * 10e6 + 2``. + +The callable object approach has the disadvantage that it is a bit slower and +results in slightly longer code. However, note that a collection of callables +can share their signature via inheritance:: + + class exponential(linear): + # __init__ inherited + def __call__(self, x): + return self.a * (x ** self.b) + +Object can encapsulate state for several methods:: + + class counter: + + value = 0 + + def set(self, x): + self.value = x + + def up(self): + self.value = self.value + 1 + + def down(self): + self.value = self.value - 1 + + count = counter() + inc, dec, reset = count.up, count.down, count.set + +Here ``inc()``, ``dec()`` and ``reset()`` act like functions which share the +same counting variable. + + +How do I copy an object in Python? +---------------------------------- + +In general, try :func:`copy.copy` or :func:`copy.deepcopy` for the general case. +Not all objects can be copied, but most can. + +Some objects can be copied more easily. Dictionaries have a :meth:`~dict.copy` +method:: + + newdict = olddict.copy() + +Sequences can be copied by slicing:: + + new_l = l[:] + + +How can I find the methods or attributes of an object? +------------------------------------------------------ + +For an instance x of a user-defined class, ``dir(x)`` returns an alphabetized +list of the names containing the instance attributes and methods and attributes +defined by its class. + + +How can my code discover the name of an object? +----------------------------------------------- + +Generally speaking, it can't, because objects don't really have names. +Essentially, assignment always binds a name to a value; The same is true of +``def`` and ``class`` statements, but in that case the value is a +callable. Consider the following code:: + + class A: + pass + + B = A + + a = B() + b = a + print b + <__main__.A instance at 0x16D07CC> + print a + <__main__.A instance at 0x16D07CC> + +Arguably the class has a name: even though it is bound to two names and invoked +through the name B the created instance is still reported as an instance of +class A. However, it is impossible to say whether the instance's name is a or +b, since both names are bound to the same value. + +Generally speaking it should not be necessary for your code to "know the names" +of particular values. Unless you are deliberately writing introspective +programs, this is usually an indication that a change of approach might be +beneficial. + +In comp.lang.python, Fredrik Lundh once gave an excellent analogy in answer to +this question: + + The same way as you get the name of that cat you found on your porch: the cat + (object) itself cannot tell you its name, and it doesn't really care -- so + the only way to find out what it's called is to ask all your neighbours + (namespaces) if it's their cat (object)... + + ....and don't be surprised if you'll find that it's known by many names, or + no name at all! + + +What's up with the comma operator's precedence? +----------------------------------------------- + +Comma is not an operator in Python. Consider this session:: + + >>> "a" in "b", "a" + (False, 'a') + +Since the comma is not an operator, but a separator between expressions the +above is evaluated as if you had entered:: + + >>> ("a" in "b"), "a" + +not:: + + >>> "a" in ("b", "a") + +The same is true of the various assignment operators (``=``, ``+=`` etc). They +are not truly operators but syntactic delimiters in assignment statements. + + +Is there an equivalent of C's "?:" ternary operator? +---------------------------------------------------- + +Yes, this feature was added in Python 2.5. The syntax would be as follows:: + + [on_true] if [expression] else [on_false] + + x, y = 50, 25 + + small = x if x < y else y + +For versions previous to 2.5 the answer would be 'No'. + +.. XXX remove rest? + +In many cases you can mimic ``a ? b : c`` with ``a and b or c``, but there's a +flaw: if *b* is zero (or empty, or ``None`` -- anything that tests false) then +*c* will be selected instead. In many cases you can prove by looking at the +code that this can't happen (e.g. because *b* is a constant or has a type that +can never be false), but in general this can be a problem. + +Tim Peters (who wishes it was Steve Majewski) suggested the following solution: +``(a and [b] or [c])[0]``. Because ``[b]`` is a singleton list it is never +false, so the wrong path is never taken; then applying ``[0]`` to the whole +thing gets the *b* or *c* that you really wanted. Ugly, but it gets you there +in the rare cases where it is really inconvenient to rewrite your code using +'if'. + +The best course is usually to write a simple ``if...else`` statement. Another +solution is to implement the ``?:`` operator as a function:: + + def q(cond, on_true, on_false): + if cond: + if not isfunction(on_true): + return on_true + else: + return on_true() + else: + if not isfunction(on_false): + return on_false + else: + return on_false() + +In most cases you'll pass b and c directly: ``q(a, b, c)``. To avoid evaluating +b or c when they shouldn't be, encapsulate them within a lambda function, e.g.: +``q(a, lambda: b, lambda: c)``. + +It has been asked *why* Python has no if-then-else expression. There are +several answers: many languages do just fine without one; it can easily lead to +less readable code; no sufficiently "Pythonic" syntax has been discovered; a +search of the standard library found remarkably few places where using an +if-then-else expression would make the code more understandable. + +In 2002, :pep:`308` was written proposing several possible syntaxes and the +community was asked to vote on the issue. The vote was inconclusive. Most +people liked one of the syntaxes, but also hated other syntaxes; many votes +implied that people preferred no ternary operator rather than having a syntax +they hated. + + +Is it possible to write obfuscated one-liners in Python? +-------------------------------------------------------- + +Yes. Usually this is done by nesting :keyword:`lambda` within +:keyword:`lambda`. See the following three examples, due to Ulf Bartelt:: + + # Primes < 1000 + print filter(None,map(lambda y:y*reduce(lambda x,y:x*y!=0, + map(lambda x,y=y:y%x,range(2,int(pow(y,0.5)+1))),1),range(2,1000))) + + # First 10 Fibonacci numbers + print map(lambda x,f=lambda x,f:(f(x-1,f)+f(x-2,f)) if x>1 else 1: f(x,f), + range(10)) + + # Mandelbrot set + print (lambda Ru,Ro,Iu,Io,IM,Sx,Sy:reduce(lambda x,y:x+y,map(lambda y, + Iu=Iu,Io=Io,Ru=Ru,Ro=Ro,Sy=Sy,L=lambda yc,Iu=Iu,Io=Io,Ru=Ru,Ro=Ro,i=IM, + Sx=Sx,Sy=Sy:reduce(lambda x,y:x+y,map(lambda x,xc=Ru,yc=yc,Ru=Ru,Ro=Ro, + i=i,Sx=Sx,F=lambda xc,yc,x,y,k,f=lambda xc,yc,x,y,k,f:(k<=0)or (x*x+y*y + >=4.0) or 1+f(xc,yc,x*x-y*y+xc,2.0*x*y+yc,k-1,f):f(xc,yc,x,y,k,f):chr( + 64+F(Ru+x*(Ro-Ru)/Sx,yc,0,0,i)),range(Sx))):L(Iu+y*(Io-Iu)/Sy),range(Sy + ))))(-2.1, 0.7, -1.2, 1.2, 30, 80, 24) + # \___ ___/ \___ ___/ | | |__ lines on screen + # V V | |______ columns on screen + # | | |__________ maximum of "iterations" + # | |_________________ range on y axis + # |____________________________ range on x axis + +Don't try this at home, kids! + + +Numbers and strings +=================== + +How do I specify hexadecimal and octal integers? +------------------------------------------------ + +To specify an octal digit, precede the octal value with a zero, and then a lower +or uppercase "o". For example, to set the variable "a" to the octal value "10" +(8 in decimal), type:: + + >>> a = 0o10 + >>> a + 8 + +Hexadecimal is just as easy. Simply precede the hexadecimal number with a zero, +and then a lower or uppercase "x". Hexadecimal digits can be specified in lower +or uppercase. For example, in the Python interpreter:: + + >>> a = 0xa5 + >>> a + 165 + >>> b = 0XB2 + >>> b + 178 + + +Why does -22 // 10 return -3? +----------------------------- + +It's primarily driven by the desire that ``i % j`` have the same sign as ``j``. +If you want that, and also want:: + + i == (i // j) * j + (i % j) + +then integer division has to return the floor. C also requires that identity to +hold, and then compilers that truncate ``i // j`` need to make ``i % j`` have +the same sign as ``i``. + +There are few real use cases for ``i % j`` when ``j`` is negative. When ``j`` +is positive, there are many, and in virtually all of them it's more useful for +``i % j`` to be ``>= 0``. If the clock says 10 now, what did it say 200 hours +ago? ``-190 % 12 == 2`` is useful; ``-190 % 12 == -10`` is a bug waiting to +bite. + +.. note:: + + On Python 2, ``a / b`` returns the same as ``a // b`` if + ``__future__.division`` is not in effect. This is also known as "classic" + division. + + +How do I convert a string to a number? +-------------------------------------- + +For integers, use the built-in :func:`int` type constructor, e.g. ``int('144') +== 144``. Similarly, :func:`float` converts to floating-point, +e.g. ``float('144') == 144.0``. + +By default, these interpret the number as decimal, so that ``int('0144') == +144`` and ``int('0x144')`` raises :exc:`ValueError`. ``int(string, base)`` takes +the base to convert from as a second optional argument, so ``int('0x144', 16) == +324``. If the base is specified as 0, the number is interpreted using Python's +rules: a leading '0' indicates octal, and '0x' indicates a hex number. + +Do not use the built-in function :func:`eval` if all you need is to convert +strings to numbers. :func:`eval` will be significantly slower and it presents a +security risk: someone could pass you a Python expression that might have +unwanted side effects. For example, someone could pass +``__import__('os').system("rm -rf $HOME")`` which would erase your home +directory. + +:func:`eval` also has the effect of interpreting numbers as Python expressions, +so that e.g. ``eval('09')`` gives a syntax error because Python regards numbers +starting with '0' as octal (base 8). + + +How do I convert a number to a string? +-------------------------------------- + +To convert, e.g., the number 144 to the string '144', use the built-in type +constructor :func:`str`. If you want a hexadecimal or octal representation, use +the built-in functions :func:`hex` or :func:`oct`. For fancy formatting, see +the :ref:`formatstrings` section, e.g. ``"{:04d}".format(144)`` yields +``'0144'`` and ``"{:.3f}".format(1/3)`` yields ``'0.333'``. You may also use +:ref:`the % operator <string-formatting>` on strings. See the library reference +manual for details. + + +How do I modify a string in place? +---------------------------------- + +You can't, because strings are immutable. If you need an object with this +ability, try converting the string to a list or use the array module:: + + >>> s = "Hello, world" + >>> a = list(s) + >>> print a + ['H', 'e', 'l', 'l', 'o', ',', ' ', 'w', 'o', 'r', 'l', 'd'] + >>> a[7:] = list("there!") + >>> ''.join(a) + 'Hello, there!' + + >>> import array + >>> a = array.array('c', s) + >>> print a + array('c', 'Hello, world') + >>> a[0] = 'y' ; print a + array('c', 'yello world') + >>> a.tostring() + 'yello, world' + + +How do I use strings to call functions/methods? +----------------------------------------------- + +There are various techniques. + +* The best is to use a dictionary that maps strings to functions. The primary + advantage of this technique is that the strings do not need to match the names + of the functions. This is also the primary technique used to emulate a case + construct:: + + def a(): + pass + + def b(): + pass + + dispatch = {'go': a, 'stop': b} # Note lack of parens for funcs + + dispatch[get_input()]() # Note trailing parens to call function + +* Use the built-in function :func:`getattr`:: + + import foo + getattr(foo, 'bar')() + + Note that :func:`getattr` works on any object, including classes, class + instances, modules, and so on. + + This is used in several places in the standard library, like this:: + + class Foo: + def do_foo(self): + ... + + def do_bar(self): + ... + + f = getattr(foo_instance, 'do_' + opname) + f() + + +* Use :func:`locals` or :func:`eval` to resolve the function name:: + + def myFunc(): + print "hello" + + fname = "myFunc" + + f = locals()[fname] + f() + + f = eval(fname) + f() + + Note: Using :func:`eval` is slow and dangerous. If you don't have absolute + control over the contents of the string, someone could pass a string that + resulted in an arbitrary function being executed. + +Is there an equivalent to Perl's chomp() for removing trailing newlines from strings? +------------------------------------------------------------------------------------- + +Starting with Python 2.2, you can use ``S.rstrip("\r\n")`` to remove all +occurences of any line terminator from the end of the string ``S`` without +removing other trailing whitespace. If the string ``S`` represents more than +one line, with several empty lines at the end, the line terminators for all the +blank lines will be removed:: + + >>> lines = ("line 1 \r\n" + ... "\r\n" + ... "\r\n") + >>> lines.rstrip("\n\r") + 'line 1 ' + +Since this is typically only desired when reading text one line at a time, using +``S.rstrip()`` this way works well. + +For older versions of Python, there are two partial substitutes: + +- If you want to remove all trailing whitespace, use the ``rstrip()`` method of + string objects. This removes all trailing whitespace, not just a single + newline. + +- Otherwise, if there is only one line in the string ``S``, use + ``S.splitlines()[0]``. + + +Is there a scanf() or sscanf() equivalent? +------------------------------------------ + +Not as such. + +For simple input parsing, the easiest approach is usually to split the line into +whitespace-delimited words using the :meth:`~str.split` method of string objects +and then convert decimal strings to numeric values using :func:`int` or +:func:`float`. ``split()`` supports an optional "sep" parameter which is useful +if the line uses something other than whitespace as a separator. + +For more complicated input parsing, regular expressions more powerful than C's +:cfunc:`sscanf` and better suited for the task. + + +What does 'UnicodeError: ASCII [decoding,encoding] error: ordinal not in range(128)' mean? +------------------------------------------------------------------------------------------ + +This error indicates that your Python installation can handle only 7-bit ASCII +strings. There are a couple ways to fix or work around the problem. + +If your programs must handle data in arbitrary character set encodings, the +environment the application runs in will generally identify the encoding of the +data it is handing you. You need to convert the input to Unicode data using +that encoding. For example, a program that handles email or web input will +typically find character set encoding information in Content-Type headers. This +can then be used to properly convert input data to Unicode. Assuming the string +referred to by ``value`` is encoded as UTF-8:: ***The diff for this file has been truncated for email.*** ======================================= --- /dev/null +++ /faq/python-video-icon.png Sat Mar 19 08:59:41 2011 Binary file, no diff available. ======================================= --- /dev/null +++ /faq/windows.rst Sat Mar 19 08:59:41 2011 @@ -0,0 +1,609 @@ +:tocdepth: 2 + +.. _windows-faq: + +===================== +Python on Windows FAQ +===================== + +.. contents:: + +How do I run a Python program under Windows? +-------------------------------------------- + +This is not necessarily a straightforward question. If you are already familiar +with running programs from the Windows command line then everything will seem +obvious; otherwise, you might need a little more guidance. There are also +differences between Windows 95, 98, NT, ME, 2000 and XP which can add to the +confusion. + +.. sidebar:: |Python Development on XP|_ + :subtitle: `Python Development on XP`_ + + This series of screencasts aims to get you up and running with Python on + Windows XP. The knowledge is distilled into 1.5 hours and will get you up + and running with the right Python distribution, coding in your choice of IDE, + and debugging and writing solid code with unit-tests. + +.. |Python Development on XP| image:: python-video-icon.png +.. _`Python Development on XP`: + http://www.showmedo.com/videos/series?name=pythonOzsvaldPyNewbieSeries + +Unless you use some sort of integrated development environment, you will end up +*typing* Windows commands into what is variously referred to as a "DOS window" +or "Command prompt window". Usually you can create such a window from your +Start menu; under Windows 2000 the menu selection is :menuselection:`Start --> +Programs --> Accessories --> Command Prompt`. You should be able to recognize +when you have started such a window because you will see a Windows "command +prompt", which usually looks like this:: + + C:\> + +The letter may be different, and there might be other things after it, so you +might just as easily see something like:: + + D:\Steve\Projects\Python> + +depending on how your computer has been set up and what else you have recently +done with it. Once you have started such a window, you are well on the way to +running Python programs. + +You need to realize that your Python scripts have to be processed by another +program called the Python interpreter. The interpreter reads your script, +compiles it into bytecodes, and then executes the bytecodes to run your +program. So, how do you arrange for the interpreter to handle your Python? + +First, you need to make sure that your command window recognises the word +"python" as an instruction to start the interpreter. If you have opened a +command window, you should try entering the command ``python`` and hitting +return. You should then see something like:: + + Python 2.2 (#28, Dec 21 2001, 12:21:22) [MSC 32 bit (Intel)] on win32 + Type "help", "copyright", "credits" or "license" for more information. + >>> + +You have started the interpreter in "interactive mode". That means you can enter +Python statements or expressions interactively and have them executed or +evaluated while you wait. This is one of Python's strongest features. Check it +by entering a few expressions of your choice and seeing the results:: + + >>> print "Hello" + Hello + >>> "Hello" * 3 + HelloHelloHello + +Many people use the interactive mode as a convenient yet highly programmable +calculator. When you want to end your interactive Python session, hold the Ctrl +key down while you enter a Z, then hit the "Enter" key to get back to your +Windows command prompt. + +You may also find that you have a Start-menu entry such as :menuselection:`Start +--> Programs --> Python 2.2 --> Python (command line)` that results in you +seeing the ``>>>`` prompt in a new window. If so, the window will disappear +after you enter the Ctrl-Z character; Windows is running a single "python" +command in the window, and closes it when you terminate the interpreter. + +If the ``python`` command, instead of displaying the interpreter prompt ``>>>``, +gives you a message like:: + + 'python' is not recognized as an internal or external command, + operable program or batch file. + +.. sidebar:: |Adding Python to DOS Path|_ + :subtitle: `Adding Python to DOS Path`_ + + Python is not added to the DOS path by default. This screencast will walk + you through the steps to add the correct entry to the `System Path`, allowing + Python to be executed from the command-line by all users. + +.. |Adding Python to DOS Path| image:: python-video-icon.png +.. _`Adding Python to DOS Path`: + http://showmedo.com/videos/video?name=960000&fromSeriesID=96 + + +or:: + + Bad command or filename + +then you need to make sure that your computer knows where to find the Python +interpreter. To do this you will have to modify a setting called PATH, which is +a list of directories where Windows will look for programs. + +You should arrange for Python's installation directory to be added to the PATH +of every command window as it starts. If you installed Python fairly recently +then the command :: + + dir C:\py* + +will probably tell you where it is installed; the usual location is something +like ``C:\Python23``. Otherwise you will be reduced to a search of your whole +disk ... use :menuselection:`Tools --> Find` or hit the :guilabel:`Search` +button and look for "python.exe". Supposing you discover that Python is +installed in the ``C:\Python23`` directory (the default at the time of writing), +you should make sure that entering the command :: + + c:\Python23\python + +starts up the interpreter as above (and don't forget you'll need a "CTRL-Z" and +an "Enter" to get out of it). Once you have verified the directory, you need to +add it to the start-up routines your computer goes through. For older versions +of Windows the easiest way to do this is to edit the ``C:\AUTOEXEC.BAT`` +file. You would want to add a line like the following to ``AUTOEXEC.BAT``:: + + PATH C:\Python23;%PATH% + +For Windows NT, 2000 and (I assume) XP, you will need to add a string such as :: + + ;C:\Python23 + +to the current setting for the PATH environment variable, which you will find in +the properties window of "My Computer" under the "Advanced" tab. Note that if +you have sufficient privilege you might get a choice of installing the settings +either for the Current User or for System. The latter is preferred if you want +everybody to be able to run Python on the machine. + +If you aren't confident doing any of these manipulations yourself, ask for help! +At this stage you may want to reboot your system to make absolutely sure the new +setting has taken effect. You probably won't need to reboot for Windows NT, XP +or 2000. You can also avoid it in earlier versions by editing the file +``C:\WINDOWS\COMMAND\CMDINIT.BAT`` instead of ``AUTOEXEC.BAT``. + +You should now be able to start a new command window, enter ``python`` at the +``C:\>`` (or whatever) prompt, and see the ``>>>`` prompt that indicates the +Python interpreter is reading interactive commands. + +Let's suppose you have a program called ``pytest.py`` in directory +``C:\Steve\Projects\Python``. A session to run that program might look like +this:: + + C:\> cd \Steve\Projects\Python + C:\Steve\Projects\Python> python pytest.py + +Because you added a file name to the command to start the interpreter, when it +starts up it reads the Python script in the named file, compiles it, executes +it, and terminates, so you see another ``C:\>`` prompt. You might also have +entered :: + + C:\> python \Steve\Projects\Python\pytest.py + +if you hadn't wanted to change your current directory. + +Under NT, 2000 and XP you may well find that the installation process has also +arranged that the command ``pytest.py`` (or, if the file isn't in the current +directory, ``C:\Steve\Projects\Python\pytest.py``) will automatically recognize +the ".py" extension and run the Python interpreter on the named file. Using this +feature is fine, but *some* versions of Windows have bugs which mean that this +form isn't exactly equivalent to using the interpreter explicitly, so be +careful. + +The important things to remember are: + +1. Start Python from the Start Menu, or make sure the PATH is set correctly so + Windows can find the Python interpreter. :: + + python + + should give you a '>>>' prompt from the Python interpreter. Don't forget the + CTRL-Z and ENTER to terminate the interpreter (and, if you started the window + from the Start Menu, make the window disappear). + +2. Once this works, you run programs with commands:: + + python {program-file} + +3. When you know the commands to use you can build Windows shortcuts to run the + Python interpreter on any of your scripts, naming particular working + directories, and adding them to your menus. Take a look at :: + + python --help + + if your needs are complex. + +4. Interactive mode (where you see the ``>>>`` prompt) is best used for checking + that individual statements and expressions do what you think they will, and + for developing code by experiment. + + +How do I make Python scripts executable? +---------------------------------------- + +On Windows 2000, the standard Python installer already associates the .py +extension with a file type (Python.File) and gives that file type an open +command that runs the interpreter (``D:\Program Files\Python\python.exe "%1" +%*``). This is enough to make scripts executable from the command prompt as +'foo.py'. If you'd rather be able to execute the script by simple typing 'foo' +with no extension you need to add .py to the PATHEXT environment variable. + +On Windows NT, the steps taken by the installer as described above allow you to +run a script with 'foo.py', but a longtime bug in the NT command processor +prevents you from redirecting the input or output of any script executed in this +way. This is often important. + +The incantation for making a Python script executable under WinNT is to give the +file an extension of .cmd and add the following as the first line:: + + @setlocal enableextensions & python -x %~f0 %* & goto :EOF + + +Why does Python sometimes take so long to start? +------------------------------------------------ + +Usually Python starts very quickly on Windows, but occasionally there are bug +reports that Python suddenly begins to take a long time to start up. This is +made even more puzzling because Python will work fine on other Windows systems +which appear to be configured identically. + +The problem may be caused by a misconfiguration of virus checking software on +the problem machine. Some virus scanners have been known to introduce startup +overhead of two orders of magnitude when the scanner is configured to monitor +all reads from the filesystem. Try checking the configuration of virus scanning +software on your systems to ensure that they are indeed configured identically. +McAfee, when configured to scan all file system read activity, is a particular +offender. + + +Where is Freeze for Windows? +---------------------------- + +"Freeze" is a program that allows you to ship a Python program as a single +stand-alone executable file. It is *not* a compiler; your programs don't run +any faster, but they are more easily distributable, at least to platforms with +the same OS and CPU. Read the README file of the freeze program for more +disclaimers. + +You can use freeze on Windows, but you must download the source tree (see +http://www.python.org/download/source). The freeze program is in the +``Tools\freeze`` subdirectory of the source tree. + +You need the Microsoft VC++ compiler, and you probably need to build Python. +The required project files are in the PCbuild directory. + + +Is a ``*.pyd`` file the same as a DLL? +-------------------------------------- + +.. XXX update for py3k (PyInit_foo) + +Yes, .pyd files are dll's, but there are a few differences. If you have a DLL +named ``foo.pyd``, then it must have a function ``initfoo()``. You can then +write Python "import foo", and Python will search for foo.pyd (as well as +foo.py, foo.pyc) and if it finds it, will attempt to call ``initfoo()`` to +initialize it. You do not link your .exe with foo.lib, as that would cause +Windows to require the DLL to be present. + +Note that the search path for foo.pyd is PYTHONPATH, not the same as the path +that Windows uses to search for foo.dll. Also, foo.pyd need not be present to +run your program, whereas if you linked your program with a dll, the dll is +required. Of course, foo.pyd is required if you want to say ``import foo``. In +a DLL, linkage is declared in the source code with ``__declspec(dllexport)``. +In a .pyd, linkage is defined in a list of available functions. + + +How can I embed Python into a Windows application? +-------------------------------------------------- + +Embedding the Python interpreter in a Windows app can be summarized as follows: + +1. Do _not_ build Python into your .exe file directly. On Windows, Python must + be a DLL to handle importing modules that are themselves DLL's. (This is the + first key undocumented fact.) Instead, link to :file:`python{NN}.dll`; it is + typically installed in ``C:\Windows\System``. NN is the Python version, a + number such as "23" for Python 2.3. + + You can link to Python statically or dynamically. Linking statically means + linking against :file:`python{NN}.lib`, while dynamically linking means + linking against :file:`python{NN}.dll`. The drawback to dynamic linking is + that your app won't run if :file:`python{NN}.dll` does not exist on your + system. (General note: :file:`python{NN}.lib` is the so-called "import lib" + corresponding to :file:`python.dll`. It merely defines symbols for the + linker.) + + Linking dynamically greatly simplifies link options; everything happens at + run time. Your code must load :file:`python{NN}.dll` using the Windows + ``LoadLibraryEx()`` routine. The code must also use access routines and data + in :file:`python{NN}.dll` (that is, Python's C API's) using pointers obtained + by the Windows ``GetProcAddress()`` routine. Macros can make using these + pointers transparent to any C code that calls routines in Python's C API. + + Borland note: convert :file:`python{NN}.lib` to OMF format using Coff2Omf.exe + first. + +2. If you use SWIG, it is easy to create a Python "extension module" that will + make the app's data and methods available to Python. SWIG will handle just + about all the grungy details for you. The result is C code that you link + *into* your .exe file (!) You do _not_ have to create a DLL file, and this + also simplifies linking. + +3. SWIG will create an init function (a C function) whose name depends on the + name of the extension module. For example, if the name of the module is leo, + the init function will be called initleo(). If you use SWIG shadow classes, + as you should, the init function will be called initleoc(). This initializes + a mostly hidden helper class used by the shadow class. + + The reason you can link the C code in step 2 into your .exe file is that + calling the initialization function is equivalent to importing the module + into Python! (This is the second key undocumented fact.) + +4. In short, you can use the following code to initialize the Python interpreter + with your extension module. + + .. code-block:: c + + #include "python.h" + ... + Py_Initialize(); // Initialize Python. + initmyAppc(); // Initialize (import) the helper class. + PyRun_SimpleString("import myApp") ; // Import the shadow class. + +5. There are two problems with Python's C API which will become apparent if you + use a compiler other than MSVC, the compiler used to build pythonNN.dll. + + Problem 1: The so-called "Very High Level" functions that take FILE * + arguments will not work in a multi-compiler environment because each + compiler's notion of a struct FILE will be different. From an implementation + standpoint these are very _low_ level functions. + + Problem 2: SWIG generates the following code when generating wrappers to void + functions: + + .. code-block:: c + + Py_INCREF(Py_None); + _resultobj = Py_None; + return _resultobj; + + Alas, Py_None is a macro that expands to a reference to a complex data + structure called _Py_NoneStruct inside pythonNN.dll. Again, this code will + fail in a mult-compiler environment. Replace such code by: + + .. code-block:: c + + return Py_BuildValue(""); + + It may be possible to use SWIG's ``%typemap`` command to make the change + automatically, though I have not been able to get this to work (I'm a + complete SWIG newbie). + +6. Using a Python shell script to put up a Python interpreter window from inside + your Windows app is not a good idea; the resulting window will be independent + of your app's windowing system. Rather, you (or the wxPythonWindow class) + should create a "native" interpreter window. It is easy to connect that + window to the Python interpreter. You can redirect Python's i/o to _any_ + object that supports read and write, so all you need is a Python object + (defined in your extension module) that contains read() and write() methods. + + +How do I use Python for CGI? +---------------------------- + +On the Microsoft IIS server or on the Win95 MS Personal Web Server you set up +Python in the same way that you would set up any other scripting engine. + +Run regedt32 and go to:: + + HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W3SVC\Parameters\ScriptMap + +and enter the following line (making any specific changes that your system may +need):: + + .py :REG_SZ: c:\<path to python>\python.exe -u %s %s + +This line will allow you to call your script with a simple reference like: +``http://yourserver/scripts/yourscript.py`` provided "scripts" is an +"executable" directory for your server (which it usually is by default). The +:option:`-u` flag specifies unbuffered and binary mode for stdin - needed when +working with binary data. + +In addition, it is recommended that using ".py" may not be a good idea for the +file extensions when used in this context (you might want to reserve ``*.py`` +for support modules and use ``*.cgi`` or ``*.cgp`` for "main program" scripts). + +In order to set up Internet Information Services 5 to use Python for CGI +processing, please see the following links: + + http://www.e-coli.net/pyiis_server.html (for Win2k Server) + http://www.e-coli.net/pyiis.html (for Win2k pro) + +Configuring Apache is much simpler. In the Apache configuration file +``httpd.conf``, add the following line at the end of the file:: + + ScriptInterpreterSource Registry + +Then, give your Python CGI-scripts the extension .py and put them in the cgi-bin +directory. + + +How do I keep editors from inserting tabs into my Python source? +---------------------------------------------------------------- + +The FAQ does not recommend using tabs, and the Python style guide, :pep:`8`, +recommends 4 spaces for distributed Python code; this is also the Emacs +python-mode default. + +Under any editor, mixing tabs and spaces is a bad idea. MSVC is no different in +this respect, and is easily configured to use spaces: Take :menuselection:`Tools +--> Options --> Tabs`, and for file type "Default" set "Tab size" and "Indent +size" to 4, and select the "Insert spaces" radio button. + +If you suspect mixed tabs and spaces are causing problems in leading whitespace, +run Python with the :option:`-t` switch or run ``Tools/Scripts/tabnanny.py`` to +check a directory tree in batch mode. + + +How do I check for a keypress without blocking? +----------------------------------------------- + +Use the msvcrt module. This is a standard Windows-specific extension module. +It defines a function ``kbhit()`` which checks whether a keyboard hit is +present, and ``getch()`` which gets one character without echoing it. + + +How do I emulate os.kill() in Windows? +-------------------------------------- + +To terminate a process, you can use ctypes:: + + import ctypes + + def kill(pid): + """kill function for Win32""" + kernel32 = ctypes.windll.kernel32 + handle = kernel32.OpenProcess(1, 0, pid) + return (0 != kernel32.TerminateProcess(handle, 0)) + + +Why does os.path.isdir() fail on NT shared directories? +------------------------------------------------------- + +The solution appears to be always append the "\\" on the end of shared +drives. + + >>> import os + >>> os.path.isdir( '\\\\rorschach\\public') + 0 + >>> os.path.isdir( '\\\\rorschach\\public\\') + 1 + +It helps to think of share points as being like drive letters. Example:: + + k: is not a directory + k:\ is a directory + k:\media is a directory + k:\media\ is not a directory + +The same rules apply if you substitute "k:" with "\\conky\foo":: + + \\conky\foo is not a directory + \\conky\foo\ is a directory + \\conky\foo\media is a directory + \\conky\foo\media\ is not a directory + + +cgi.py (or other CGI programming) doesn't work sometimes on NT or win95! +------------------------------------------------------------------------ + +Be sure you have the latest python.exe, that you are using python.exe rather +than a GUI version of Python and that you have configured the server to execute +:: + + "...\python.exe -u ..." + +for the CGI execution. The :option:`-u` (unbuffered) option on NT and Win95 +prevents the interpreter from altering newlines in the standard input and +output. Without it post/multipart requests will seem to have the wrong length +and binary (e.g. GIF) responses may get garbled (resulting in broken images, PDF +files, and other binary downloads failing). + + +Why doesn't os.popen() work in PythonWin on NT? +----------------------------------------------- + +The reason that os.popen() doesn't work from within PythonWin is due to a bug in +Microsoft's C Runtime Library (CRT). The CRT assumes you have a Win32 console +attached to the process. + +You should use the win32pipe module's popen() instead which doesn't depend on +having an attached Win32 console. + +Example:: + + import win32pipe + f = win32pipe.popen('dir /c c:\\') + print f.readlines() + f.close() + + +Why doesn't os.popen()/win32pipe.popen() work on Win9x? +------------------------------------------------------- + +There is a bug in Win9x that prevents os.popen/win32pipe.popen* from +working. The good news is there is a way to work around this problem. The +Microsoft Knowledge Base article that you need to lookup is: Q150956. You will +find links to the knowledge base at: http://support.microsoft.com/. + + +PyRun_SimpleFile() crashes on Windows but not on Unix; why? +----------------------------------------------------------- + +This is very sensitive to the compiler vendor, version and (perhaps) even +options. If the FILE* structure in your embedding program isn't the same as is +assumed by the Python interpreter it won't work. + +The Python 1.5.* DLLs (``python15.dll``) are all compiled with MS VC++ 5.0 and +with multithreading-DLL options (``/MD``). + +If you can't change compilers or flags, try using :cfunc:`Py_RunSimpleString`. +A trick to get it to run an arbitrary file is to construct a call to +:func:`execfile` with the name of your file as argument. + +Also note that you can not mix-and-match Debug and Release versions. If you +wish to use the Debug Multithreaded DLL, then your module *must* have an "_d" +appended to the base name. + + +Importing _tkinter fails on Windows 95/98: why? +------------------------------------------------ + +Sometimes, the import of _tkinter fails on Windows 95 or 98, complaining with a +message like the following:: + + ImportError: DLL load failed: One of the library files needed + to run this application cannot be found. + +It could be that you haven't installed Tcl/Tk, but if you did install Tcl/Tk, +and the Wish application works correctly, the problem may be that its installer +didn't manage to edit the autoexec.bat file correctly. It tries to add a +statement that changes the PATH environment variable to include the Tcl/Tk 'bin' +subdirectory, but sometimes this edit doesn't quite work. Opening it with +notepad usually reveals what the problem is. + +(One additional hint, noted by David Szafranski: you can't use long filenames +here; e.g. use ``C:\PROGRA~1\Tcl\bin`` instead of ``C:\Program Files\Tcl\bin``.) + + +How do I extract the downloaded documentation on Windows? +--------------------------------------------------------- + +Sometimes, when you download the documentation package to a Windows machine +using a web browser, the file extension of the saved file ends up being .EXE. +This is a mistake; the extension should be .TGZ. + +Simply rename the downloaded file to have the .TGZ extension, and WinZip will be +able to handle it. (If your copy of WinZip doesn't, get a newer one from +http://www.winzip.com.) + + +Missing cw3215mt.dll (or missing cw3215.dll) +-------------------------------------------- + +Sometimes, when using Tkinter on Windows, you get an error that cw3215mt.dll or +cw3215.dll is missing. + +Cause: you have an old Tcl/Tk DLL built with cygwin in your path (probably +``C:\Windows``). You must use the Tcl/Tk DLLs from the standard Tcl/Tk +installation (Python 1.5.2 comes with one). + + +Warning about CTL3D32 version from installer +-------------------------------------------- + +The Python installer issues a warning like this:: + + This version uses ``CTL3D32.DLL`` which is not the correct version. + This version is used for windows NT applications only. + +Tim Peters: + + This is a Microsoft DLL, and a notorious source of problems. The message + means what it says: you have the wrong version of this DLL for your operating + system. The Python installation did not cause this -- something else you + installed previous to this overwrote the DLL that came with your OS (probably + older shareware of some sort, but there's no way to tell now). If you search + for "CTL3D32" using any search engine (AltaVista, for example), you'll find + hundreds and hundreds of web pages complaining about the same problem with + all sorts of installation programs. They'll point you to ways to get the + correct version reinstalled on your system (since Python doesn't cause this, + we can't fix it). + +David A Burton has written a little program to fix this. Go to +http://www.burtonsys.com/downloads.html and click on "ctl3dfix.zip".
