This talk consists of three parts, it is about certain aspects
you may encounter while extending SAGE.
The first part deals with the documentation,
its organisation and how to modify it.
The second part deals with the conventions and technical aspects
of SAGE package creation.
The last part covers different smaller topics like how to use tools intended for
Python with SAGE.
The first and second part are a summary of the SAGE Programming Guide, Chapter 2, 3 ,4.
$SAGE_ROOT is an environment variable pointing to the root directory of the SAGE installation. At home it points to /usr/local/sage, on linserv it points to /usr/local/share/sage-2.8.15.
The documentation is stored in $SAGE_ROOT/devel/doc. There are sub-directories for the Tutorial (tut), the Programming Guide (prog), the Constructions (const), the Installation Guide (inst), and the Reference Manual (ref). The first four are handwritten LaTeX files, but the Reference Manual is created automatically from a manual selection.
SAGE comes with a full version control system called Mercurial and the $SAGE_ROOT/devel/doc directory is version controlled. So is any modification to the documentation: you can access repository functions from the hg_doc object within SAGE.
At first we need to configure Mercurial with our user-name. Create the file ~/.hgrc with (hg: the chemical symbol for mercury)
[ui] username = UserName <EMail-Adress>
The Programming Guide, Chapter 4.4 has an example session.
sage: hg_doc.pull() cd "/home/jaap/sage/devel/doc" && hg status cd "/home/jaap/sage/devel/doc" && hg status cd "/home/jaap/sage/devel/doc" && hg pull -u http://sagemath.org/sage//hg//doc-main pulling from http://sagemath.org/sage//hg//doc-main searching for changes adding changesets adding manifests adding file changes added 3 changesets with 3 changes to 1 files 1 files updated, 0 files merged, 0 files removed, 0 files unresolved If it says use 'hg merge' above, then you should type hg_sage.merge(), where hg_sage is the name of the repository you are using. This might not work with the notebook yet.hg_doc.pull() updates your local repository. After modifying some file you can commit the changes:
sage: hg_doc.commit() cd "/home/jaap/downloads/sage-1.6/devel/doc" && hg diff | less cd "/home/jaap/downloads/sage-1.6/devel/doc" && hg commitand create a patch-file to send to the SAGE-Developers:
sage: hg_doc.send('prog20070118.hg') Writing to /home/jaap/sage/prog20070118.hg cd "/home/jaap/sage/devel/doc" && hg bundle tmphg http://sagemath.org/sage//hg//doc-main searching for changes Successfully created hg patch bundle /home/jaap/sage/prog20070118.hg
Each directory contains build_dvi and build_pdf scripts. The command make html in $SAGE_ROOT/devel/doc rebuilds the html-documentation. It is also possible to use make pdf from $SAGE_ROOT/devel/doc. You can choose between letter and a4 in the Makefile.
The file $SAGE_ROOT/devel/doc/commontex/macros.tex defines some macros. If you need more tell William Stein what to include (or send a hg patch).
The SAGE-Code examples inside the LaTeX files should be in verbatim environments and can be tested with
sage -t filename.texTwo verbatim environments can be tested together by giving a %link comment at the end of the first and the beginning of the second environment. So some text between the code is possible.
\begin{verbatim} sage: def add7(n):return n+7 ....: \end{verbatim}%link Some Explanation %link \begin{verbatim} sage: add7(7) 14 \end{verbatim}Without %link both environments would be tested independently and add7 would be unknown in the second environment.
In $SAGE_ROOT/devel/doc/ref:
ref.tex: groups.tex rings.tex rings-standard.tex rings-padic.tex rings-numerical.tex number-fields.tex polynomial-rings.texAnd much more files are read in with \input from ref.tex. ref.tex and the files it includes, are created by hand. This determines the selection and the order of items in the Reference. The files which are inputted by ref.tex consists again of many inputs: for example structures.tex contains:
chapter{Basic Structure\label{ch:structures}} \input{sage.structure.sage-object} \input{sage.structure.parent-gens} \input{sage.structure.formal-sum} \input{sage.structure.factorization} \input{sage.structure.element} \input{sage.structure.mutability} \input{sage.structure.sequence} \input{sage.sets.set} \input{sage.sets.primes}
The files beginning with sage are created by scripts (update and update_script.py in $SAGE_ROOT/devel/doc/ref) from the doc-strings of the corresponding modules.
For example open the file $SAGE_ROOT/devel/sage/sage/structure/element.pyx and compare it with $SAGE_ROOT/devel/doc/ref/sage.structure.element.tex and Chapter "10.5 Elements" of the SAGE Reference Manual
Just change the LaTeX file, test your examples, then run build_dvi or update.
For doc-strings containing LaTeX commands, Python's raw strings are the best choice, so you can use r"\tau" instead of "\\tau".
Send a patch to William Stein.
Directory names can be plural, and file names should be singular. Package author are encouraged to include notes as plain text files in a sub-directory notes. In the context of an import statement, directory and file-name become identifiers (from the grammar):
identifier ::= (letter|"_") (letter | digit | "_")* import_stmt ::= "import" module ["as" name] ( "," module ["as" name] )* | "from" module "import" identifier ["as" name] ( "," identifier ["as" name] )* | "from" module "import" "(" identifier ["as" name] ( "," identifier ["as" name] )* [","] ")" | "from" module "import" "*" module ::= (identifier ".")* identifierwhich means that only underscore and letters and digits (but not in the first place) are allowed for file and directory names which are likely to become package and module names in the near future.
/finite-quadratic-modules/cn-group/quadratic-modules.sageto
/finite_quadratic_modules/cn_group/quadratic_modules.sage
Every code file should start with a documentation string of the following format:
r""" Short Summary Paragraph with a more detailed description AUTHORS - First author (year-month-date): initial version - other author (year-month-date): short description ... Some text in LaTeX EXAMPLES: ... """ #***************************************************************************** # Copyright (C) 2006 William Stein <wstein@gmail.com> # 2006 YOUR NAME <your email> # # Distributed under the terms of the GNU General Public License (GPL) # http://www.gnu.org/licenses/ #*****************************************************************************It is important to use the GPL or a lesser license (the Programming Guide also suggests the BSD license) and to put William Stein and yourself in the Copyright notice and yourself in the Authors log (to receive credit).
Every function should be documented with a documentation string. The following fields are suggested:
Field | Description | optional |
---|---|---|
One sentence summary followed by a blank line | no | |
INPUT | Should describe the arguments, types are descriptive, i.e. integer not int, if the argument should behave like an integer. | if there are no arguments |
OUTPUT | Description of the return value, also with descriptive type. | if nothing is returned |
EXAMPLES | Examples | no |
NOTES | References, notes, ... | yes |
AUTHORS | List of authors and date given as (year-month-day) | no |
The next example shows how to use the different fields:
def point(self, x=1, y=2): r""" # note the r for "raw string" This function returns the point $(x^5,y)$. INPUT: self -- anything special about self x -- integer (default: 1) the ... y -- integer (default: 2) the ... OUTPUT: integer -- the ... EXAMPLES: This example illustrates ... sage: A = ModuliSpace() sage: A.point(2,3) xxx We now ... sage: B = A.point(5,6) sage: xxx It is an error to ... sage: C = A.point('x',7) Traceback (most recent call last): ... TypeError: unable to convert x (=r) to an integer NOTES: This function uses the algorithm of [BCDT] to determine whether an elliptic curve E over Q is modular. ... REFERENCES: [BCDT] Breuil, Conrad, Diamond, Taylor, "Modularity ...." AUTHORS: - William Stein (2005-01-03) - First_name Last_name (yyyy-mm-dd) """
sage -t collects the examples from the doc-strings in your file, then executes the code and compares the actual output with the output recorded. Code-lines are identified by the sage: prompt, so the examples can and should be copied from an interactive SAGE session.
def add7(x): r"""Add 7 to x. EXAMPLES: sage: add7(1) 8 sage: add7(2) 9 sage: add7(3) 10 sage: add7(4) 11 """ return x+7 # sage: add7(1.3) # todo: not tested # 71
sudo sage -t DocTest.sage sage -t DocTest.sage Example 0 (line 4) [3.7 s] ---------------------------------------------------------------------- All tests passed! Total time for all tests: 3.7 secondsIn the background, the file .doctest_DocTest.py is created, it uses the Python doctest module.
I used sudo (at least for the first time), because some temporary directory in $SAGE_ROOT, needed to be created. SAGE expects that code examples run without error message. Every line with a sage: is considered not only inside the EXAMPLES section.
There are a few modifiers:
There is another possibility: If the file has defines a class Test with a random() method, this method is called on a regular basis. The method should try random inputs and pass it to the other classes and methods in the file.
mkdir quadratic_module-0.1
cp ~/Sage/finite-quadratic-modules/cn-group/quadratic-modules.sage quadratic_module-0.1/quadratic_module.sage
#!/bin/bash # Three possible locations: # 1) a local python lib, more separated from sage INSTALLPATH="$SAGE_LOCAL/lib/python/site-packages/finite_modules" # 2) under sage: #INSTALLPATH="$SAGE_ROOT/devel/sage/sage/modules" # 3) if we want to install under a new dir e.g. #INSTALLPATH="$SAGE_ROOT/devel/sage/sage/modules/finite_modules" # we need to change # $SAGE_ROOT/devel/sage/setup.py if [[ ! -d "$INSTALLPATH" ]]; then mkdir "$INSTALLPATH" # mark the dir as a package-dir touch "$INSTALLPATH/__init__.py" fi # create the py file: sage quadratic_module.sage # and copy it cp quadratic_module.py "$INSTALLPATH" # if we install under sage and not under $SAGE_LOCAL/lib/python/ # we need the next two lines: #cd $SAGE_ROOT/devel/sage/ #python setup.py install # and the effect is: # copying sage/modules/quadratic_module.py -> # build/lib.linux-i686-2.5/sage/modules # copying build/lib.linux-i686-2.5/sage/modules/quadratic_module.py -> # /usr/local/sage/local/lib/python2.5/site-packages/sage/modules # perhaps we can use # build/lib.linux-i686-2.5/sage/modules # or # /usr/local/sage/local/lib/python2.5/site-packages/sage/modules # from the beginning?and make it executable:
chmod +x quadratic_module-0.1/spkg-install
sage -pkg quadratic_module-0.1
[lf@velocity:~/SAGE/SageTalk] sudo sage -i quadratic_module-0.1.spkg [sudo] password for lf: Installing quadratic_module-0.1.spkg Calling sage-spkg on quadratic_module-0.1.spkg quadratic_module-0.1 Machine: Linux velocity 2.6.22-14-generic #1 SMP Sun Oct 14 23:05:12 GMT 2007 i686 GNU/Linux Deleting directories from past builds of previous/current versions of quadratic_module-0.1 Extracting package /home/lf/SAGE/SageTalk/quadratic_module-0.1.spkg ... -rw-r--r-- 1 lf lf 13080 2008-01-09 20:52 /home/lf/SAGE/SageTalk/quadratic_module-0.1.spkg quadratic_module-0.1/ quadratic_module-0.1/quadratic_module.sage quadratic_module-0.1/spkg-install Finished extraction **************************************************** Host system uname -a: Linux velocity 2.6.22-14-generic #1 SMP Sun Oct 14 23:05:12 GMT 2007 i686 GNU/Linux **************************************************** **************************************************** GCC Version gcc -v Es werden eingebaute Spezifikationen verwendet. Ziel: i486-linux-gnu Konfiguriert mit: ../src/configure -v --enable-languages=c,c++,fortran,objc,obj-c++,treelang --prefix=/usr --enable-shared --with-system-zlib --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --enable-nls --with-gxx-include-dir=/usr/include/c++/4.1.3 --program-suffix=-4.1 --enable-__cxa_atexit --enable-clocale=gnu --enable-libstdcxx-debug --enable-mpfr --enable-checking=release i486-linux-gnu Thread-Modell: posix gcc-Version 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2) **************************************************** 1.68user 0.31system 0:02.10elapsed 94%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+22809minor)pagefaults 0swaps Successfully installed quadratic_module-0.1 Now cleaning up tmp files. Making SAGE/Python scripts relocatable... Making script relocatable
sage: import finite_modules.quadratic_module sage: finite_modules.quadratic_module? Type: module Base Class: <type 'module'> String Form: <module 'finite_modules.quadratic_module' from '/usr/local/sage/local/lib/python2.5/site-packages/finite_modules/quadratic_module.py'> Namespace: Interactive File: /usr/local/sage/local/lib/python2.5/site-packages/finite_modules/quadratic_module.py
sudo rm -f $SAGE_ROOT/spkg/installed/quadratic_module-0.1 sudo rm -f $SAGE_ROOT/devel/sage/sage/modules/quadratic_module.py sudo rm -f $SAGE_ROOT/devel/sage/build/lib.linux-i686-2.5/sage/modules/quadratic_module.py sudo rm -f $SAGE_ROOT/local/lib/python2.5/site-packages/sage/modules/quadratic_module.py
Open questions:
Installation under $SAGE_ROOT/devel/sage/sage/ or one of its sub-directories requires setup.py install. For a new directory even modifications on setup.py are necessary.
SAGE consists of many parts held together with Python code.
3.1.1.) Advice:
If you have a tool, you want to apply to a SAGE-file, the first
thing to try is: use the Python-file.
Every time you run SAGE on
a file, say sage test.sage, SAGE creates a
corresponding Python-File test.py. The differences
between both files are:
from sage.all_cmdline import *
K.<z>=CyclotomicField(7)becomes
K=CyclotomicField(Integer(7),names=('z',)); (z,) = K._first_ngens(Integer(1))
3.1.2.) Advice: instead of the python command use the command
sage -pythonwhich runs SAGE's version of Python, which knows how to find SAGE's modules.
Now we apply the previous section on "debugging within Emacs". I will show how to use the GUD (Emacs's Grand unified debugger). And in the next part we see a short overview of different Code-Browser-Plugins for Emacs.
(Continuation of the example above.)
To Debug a Python script you would use PDB, the Python
debugger. We take an example SAGE-Script an have a look how we
can use PDB with SAGE.
## First possibility: uncomment, #import pdb #pdb.set_trace() # run sage to create the py file and open it in emacs ## then in emacs: ## C-c ! and then C-c C-c ## then n p etc in the buffer K.Store the file locally as e.g. test2.sage, run sage on it, to create test2.py. Open test2.py in emacs and then issue=CyclotomicField(7) for d in range(K.degree()): print d print K.polynomial() print z ## second possibility, nicer: # to debug, run once sage test2.sage to create the test2.py file, # then in emacs: M-x pdb # then at the next prompt enter sage -python -m pdb test2.py instead of pdb # use n,s, u and p for next, step into, up and print # and remeber z^6 in sage is z**6 in python, so use p z**6 and z**7 # use w and c for refresh and continue in the debug buffer, if pdb seems to have lost track
M-x pdb sage -python -m pdb test2.py
With compiled languages like C, you can give the -g flag during compilation. It stores debugging symbols in the binary. Then you can use gdb (from emacs: M-x gdb, be sure to select Gud, GDB-UI, Display others Windows). Judging from my limited experience during the last week, gdb-mode has more features, but it is not suited for Python and SAGE (although you can run sage under the control of gdb: sage -gdb).
Why debugging?
There are several Emacs-Plugins which show the structure of a program-file in a tree structure. Depending on the programming language one or the other is better.
ECB offers file and directory browser, a class browser and a special analyse windows which displays some context information. You can change between different layouts from the menu: ECB, Change Layout, when prompted in the minibuffer, hit tab for the available layouts. I like either left15 or leftright-analyse.
With Html or LaTeX files, ecb provides an outline of the chapters and sections.
For more information about Emacs and its configuration, please visit the Emacs-section on my private homepage or have a look at the Emacs-Wiki.
Consider the next example of some C++ code, which uses a private attribute secret and notice how easy someone can access it (at least with the knowledge of the header file or the actual source file).
#include "iostream" class UUPs { private: int secret; public: int pub; UUPs(int s, int p) {secret = s; pub= p;}; int getSecret(); int getPublic(); }; int UUPs::getSecret() { return secret; } int UUPs::getPublic() { return pub; } int main(){ UUPs One(1,1); /* The next line gives the expected error: One.secret = -1; main.cpp: In function »int main()«: main.cpp:5: Fehler: »int UUPs::secret« ist privat */ std::cout << "Object One has Pub: " << One.getPublic() << "\n"; std::cout << "Object One has Secret: " << One.getSecret() << "\n"; /* Now we modify secret and no private can protect it: Go back one unit from the memory location of pub, that is the address of secret: (&(One.pub ) - 1 ) a Pointer to One.secret and nothing can protect us from shooting in our own foot: */ *(&(One.pub ) - 1 ) = -1; std::cout << "Object One has Secret: " << One.getSecret() << "\n"; /* also: get the address of One, at location offset 0 there is our secret, we only need some typecast to int: */ *((int*)(&One)) = 17; std::cout << "Object One has Secret: " << One.getSecret() << "\n"; } /* g++ uups.cpp -o uups ./uups Object One has Pub: 1 Object One has Secret: 1 Object One has Secret: -1 Object One has Secret: 17 */