Contents of /trunk/mkinitrd-magellan/klibc/usr/dash/TOUR
Parent Directory | Revision Log
Revision 1122 -
(show annotations)
(download)
Wed Aug 18 21:11:40 2010 UTC (13 years, 8 months ago) by niro
File size: 16890 byte(s)
Wed Aug 18 21:11:40 2010 UTC (13 years, 8 months ago) by niro
File size: 16890 byte(s)
-updated to klibc-1.5.19
1 | # @(#)TOUR 8.1 (Berkeley) 5/31/93 |
2 | |
3 | NOTE -- This is the original TOUR paper distributed with ash and |
4 | does not represent the current state of the shell. It is provided anyway |
5 | since it provides helpful information for how the shell is structured, |
6 | but be warned that things have changed -- the current shell is |
7 | still under development. |
8 | |
9 | ================================================================ |
10 | |
11 | A Tour through Ash |
12 | |
13 | Copyright 1989 by Kenneth Almquist. |
14 | |
15 | |
16 | DIRECTORIES: The subdirectory bltin contains commands which can |
17 | be compiled stand-alone. The rest of the source is in the main |
18 | ash directory. |
19 | |
20 | SOURCE CODE GENERATORS: Files whose names begin with "mk" are |
21 | programs that generate source code. A complete list of these |
22 | programs is: |
23 | |
24 | program intput files generates |
25 | ------- ------------ --------- |
26 | mkbuiltins builtins builtins.h builtins.c |
27 | mkinit *.c init.c |
28 | mknodes nodetypes nodes.h nodes.c |
29 | mksignames - signames.h signames.c |
30 | mksyntax - syntax.h syntax.c |
31 | mktokens - token.h |
32 | bltin/mkexpr unary_op binary_op operators.h operators.c |
33 | |
34 | There are undoubtedly too many of these. Mkinit searches all the |
35 | C source files for entries looking like: |
36 | |
37 | INIT { |
38 | x = 1; /* executed during initialization */ |
39 | } |
40 | |
41 | RESET { |
42 | x = 2; /* executed when the shell does a longjmp |
43 | back to the main command loop */ |
44 | } |
45 | |
46 | SHELLPROC { |
47 | x = 3; /* executed when the shell runs a shell procedure */ |
48 | } |
49 | |
50 | It pulls this code out into routines which are when particular |
51 | events occur. The intent is to improve modularity by isolating |
52 | the information about which modules need to be explicitly |
53 | initialized/reset within the modules themselves. |
54 | |
55 | Mkinit recognizes several constructs for placing declarations in |
56 | the init.c file. |
57 | INCLUDE "file.h" |
58 | includes a file. The storage class MKINIT makes a declaration |
59 | available in the init.c file, for example: |
60 | MKINIT int funcnest; /* depth of function calls */ |
61 | MKINIT alone on a line introduces a structure or union declara- |
62 | tion: |
63 | MKINIT |
64 | struct redirtab { |
65 | short renamed[10]; |
66 | }; |
67 | Preprocessor #define statements are copied to init.c without any |
68 | special action to request this. |
69 | |
70 | INDENTATION: The ash source is indented in multiples of six |
71 | spaces. The only study that I have heard of on the subject con- |
72 | cluded that the optimal amount to indent is in the range of four |
73 | to six spaces. I use six spaces since it is not too big a jump |
74 | from the widely used eight spaces. If you really hate six space |
75 | indentation, use the adjind (source included) program to change |
76 | it to something else. |
77 | |
78 | EXCEPTIONS: Code for dealing with exceptions appears in |
79 | exceptions.c. The C language doesn't include exception handling, |
80 | so I implement it using setjmp and longjmp. The global variable |
81 | exception contains the type of exception. EXERROR is raised by |
82 | calling error. EXINT is an interrupt. EXSHELLPROC is an excep- |
83 | tion which is raised when a shell procedure is invoked. The pur- |
84 | pose of EXSHELLPROC is to perform the cleanup actions associated |
85 | with other exceptions. After these cleanup actions, the shell |
86 | can interpret a shell procedure itself without exec'ing a new |
87 | copy of the shell. |
88 | |
89 | INTERRUPTS: In an interactive shell, an interrupt will cause an |
90 | EXINT exception to return to the main command loop. (Exception: |
91 | EXINT is not raised if the user traps interrupts using the trap |
92 | command.) The INTOFF and INTON macros (defined in exception.h) |
93 | provide uninterruptable critical sections. Between the execution |
94 | of INTOFF and the execution of INTON, interrupt signals will be |
95 | held for later delivery. INTOFF and INTON can be nested. |
96 | |
97 | MEMALLOC.C: Memalloc.c defines versions of malloc and realloc |
98 | which call error when there is no memory left. It also defines a |
99 | stack oriented memory allocation scheme. Allocating off a stack |
100 | is probably more efficient than allocation using malloc, but the |
101 | big advantage is that when an exception occurs all we have to do |
102 | to free up the memory in use at the time of the exception is to |
103 | restore the stack pointer. The stack is implemented using a |
104 | linked list of blocks. |
105 | |
106 | STPUTC: If the stack were contiguous, it would be easy to store |
107 | strings on the stack without knowing in advance how long the |
108 | string was going to be: |
109 | p = stackptr; |
110 | *p++ = c; /* repeated as many times as needed */ |
111 | stackptr = p; |
112 | The folloing three macros (defined in memalloc.h) perform these |
113 | operations, but grow the stack if you run off the end: |
114 | STARTSTACKSTR(p); |
115 | STPUTC(c, p); /* repeated as many times as needed */ |
116 | grabstackstr(p); |
117 | |
118 | We now start a top-down look at the code: |
119 | |
120 | MAIN.C: The main routine performs some initialization, executes |
121 | the user's profile if necessary, and calls cmdloop. Cmdloop is |
122 | repeatedly parses and executes commands. |
123 | |
124 | OPTIONS.C: This file contains the option processing code. It is |
125 | called from main to parse the shell arguments when the shell is |
126 | invoked, and it also contains the set builtin. The -i and -j op- |
127 | tions (the latter turns on job control) require changes in signal |
128 | handling. The routines setjobctl (in jobs.c) and setinteractive |
129 | (in trap.c) are called to handle changes to these options. |
130 | |
131 | PARSING: The parser code is all in parser.c. A recursive des- |
132 | cent parser is used. Syntax tables (generated by mksyntax) are |
133 | used to classify characters during lexical analysis. There are |
134 | three tables: one for normal use, one for use when inside single |
135 | quotes, and one for use when inside double quotes. The tables |
136 | are machine dependent because they are indexed by character vari- |
137 | ables and the range of a char varies from machine to machine. |
138 | |
139 | PARSE OUTPUT: The output of the parser consists of a tree of |
140 | nodes. The various types of nodes are defined in the file node- |
141 | types. |
142 | |
143 | Nodes of type NARG are used to represent both words and the con- |
144 | tents of here documents. An early version of ash kept the con- |
145 | tents of here documents in temporary files, but keeping here do- |
146 | cuments in memory typically results in significantly better per- |
147 | formance. It would have been nice to make it an option to use |
148 | temporary files for here documents, for the benefit of small |
149 | machines, but the code to keep track of when to delete the tem- |
150 | porary files was complex and I never fixed all the bugs in it. |
151 | (AT&T has been maintaining the Bourne shell for more than ten |
152 | years, and to the best of my knowledge they still haven't gotten |
153 | it to handle temporary files correctly in obscure cases.) |
154 | |
155 | The text field of a NARG structure points to the text of the |
156 | word. The text consists of ordinary characters and a number of |
157 | special codes defined in parser.h. The special codes are: |
158 | |
159 | CTLVAR Variable substitution |
160 | CTLENDVAR End of variable substitution |
161 | CTLBACKQ Command substitution |
162 | CTLESC Escape next character |
163 | |
164 | A variable substitution contains the following elements: |
165 | |
166 | CTLVAR type name '=' [ alternative-text CTLENDVAR ] |
167 | |
168 | The type field is a single character specifying the type of sub- |
169 | stitution. The possible types are: |
170 | |
171 | VSNORMAL $var |
172 | VSMINUS ${var-text} |
173 | VSMINUS|VSNUL ${var:-text} |
174 | VSPLUS ${var+text} |
175 | VSPLUS|VSNUL ${var:+text} |
176 | VSQUESTION ${var?text} |
177 | VSQUESTION|VSNUL ${var:?text} |
178 | VSASSIGN ${var=text} |
179 | VSASSIGN|VSNUL ${var=text} |
180 | |
181 | The name of the variable comes next, terminated by an equals |
182 | sign. If the type is not VSNORMAL, then the text field in the |
183 | substitution follows, terminated by a CTLENDVAR byte. |
184 | |
185 | Commands in back quotes are parsed and stored in a linked list. |
186 | The locations of these commands in the string are indicated by |
187 | the CTLBACKQ character. |
188 | |
189 | The character CTLESC escapes the next character, so that in case |
190 | any of the CTL characters mentioned above appear in the input, |
191 | they can be passed through transparently. CTLESC is also used to |
192 | escape '*', '?', '[', and '!' characters which were quoted by the |
193 | user and thus should not be used for file name generation. |
194 | |
195 | CTLESC characters have proved to be particularly tricky to get |
196 | right. In the case of here documents which are not subject to |
197 | variable and command substitution, the parser doesn't insert any |
198 | CTLESC characters to begin with (so the contents of the text |
199 | field can be written without any processing). Other here docu- |
200 | ments, and words which are not subject to splitting and file name |
201 | generation, have the CTLESC characters removed during the vari- |
202 | able and command substitution phase. Words which are subject |
203 | splitting and file name generation have the CTLESC characters re- |
204 | moved as part of the file name phase. |
205 | |
206 | EXECUTION: Command execution is handled by the following files: |
207 | eval.c The top level routines. |
208 | redir.c Code to handle redirection of input and output. |
209 | jobs.c Code to handle forking, waiting, and job control. |
210 | exec.c Code to to path searches and the actual exec sys call. |
211 | expand.c Code to evaluate arguments. |
212 | var.c Maintains the variable symbol table. Called from expand.c. |
213 | |
214 | EVAL.C: Evaltree recursively executes a parse tree. The exit |
215 | status is returned in the global variable exitstatus. The alter- |
216 | native entry evalbackcmd is called to evaluate commands in back |
217 | quotes. It saves the result in memory if the command is a buil- |
218 | tin; otherwise it forks off a child to execute the command and |
219 | connects the standard output of the child to a pipe. |
220 | |
221 | JOBS.C: To create a process, you call makejob to return a job |
222 | structure, and then call forkshell (passing the job structure as |
223 | an argument) to create the process. Waitforjob waits for a job |
224 | to complete. These routines take care of process groups if job |
225 | control is defined. |
226 | |
227 | REDIR.C: Ash allows file descriptors to be redirected and then |
228 | restored without forking off a child process. This is accom- |
229 | plished by duplicating the original file descriptors. The redir- |
230 | tab structure records where the file descriptors have be dupli- |
231 | cated to. |
232 | |
233 | EXEC.C: The routine find_command locates a command, and enters |
234 | the command in the hash table if it is not already there. The |
235 | third argument specifies whether it is to print an error message |
236 | if the command is not found. (When a pipeline is set up, |
237 | find_command is called for all the commands in the pipeline be- |
238 | fore any forking is done, so to get the commands into the hash |
239 | table of the parent process. But to make command hashing as |
240 | transparent as possible, we silently ignore errors at that point |
241 | and only print error messages if the command cannot be found |
242 | later.) |
243 | |
244 | The routine shellexec is the interface to the exec system call. |
245 | |
246 | EXPAND.C: Arguments are processed in three passes. The first |
247 | (performed by the routine argstr) performs variable and command |
248 | substitution. The second (ifsbreakup) performs word splitting |
249 | and the third (expandmeta) performs file name generation. If the |
250 | "/u" directory is simulated, then when "/u/username" is replaced |
251 | by the user's home directory, the flag "didudir" is set. This |
252 | tells the cd command that it should print out the directory name, |
253 | just as it would if the "/u" directory were implemented using |
254 | symbolic links. |
255 | |
256 | VAR.C: Variables are stored in a hash table. Probably we should |
257 | switch to extensible hashing. The variable name is stored in the |
258 | same string as the value (using the format "name=value") so that |
259 | no string copying is needed to create the environment of a com- |
260 | mand. Variables which the shell references internally are preal- |
261 | located so that the shell can reference the values of these vari- |
262 | ables without doing a lookup. |
263 | |
264 | When a program is run, the code in eval.c sticks any environment |
265 | variables which precede the command (as in "PATH=xxx command") in |
266 | the variable table as the simplest way to strip duplicates, and |
267 | then calls "environment" to get the value of the environment. |
268 | There are two consequences of this. First, if an assignment to |
269 | PATH precedes the command, the value of PATH before the assign- |
270 | ment must be remembered and passed to shellexec. Second, if the |
271 | program turns out to be a shell procedure, the strings from the |
272 | environment variables which preceded the command must be pulled |
273 | out of the table and replaced with strings obtained from malloc, |
274 | since the former will automatically be freed when the stack (see |
275 | the entry on memalloc.c) is emptied. |
276 | |
277 | BUILTIN COMMANDS: The procedures for handling these are scat- |
278 | tered throughout the code, depending on which location appears |
279 | most appropriate. They can be recognized because their names al- |
280 | ways end in "cmd". The mapping from names to procedures is |
281 | specified in the file builtins, which is processed by the mkbuil- |
282 | tins command. |
283 | |
284 | A builtin command is invoked with argc and argv set up like a |
285 | normal program. A builtin command is allowed to overwrite its |
286 | arguments. Builtin routines can call nextopt to do option pars- |
287 | ing. This is kind of like getopt, but you don't pass argc and |
288 | argv to it. Builtin routines can also call error. This routine |
289 | normally terminates the shell (or returns to the main command |
290 | loop if the shell is interactive), but when called from a builtin |
291 | command it causes the builtin command to terminate with an exit |
292 | status of 2. |
293 | |
294 | The directory bltins contains commands which can be compiled in- |
295 | dependently but can also be built into the shell for efficiency |
296 | reasons. The makefile in this directory compiles these programs |
297 | in the normal fashion (so that they can be run regardless of |
298 | whether the invoker is ash), but also creates a library named |
299 | bltinlib.a which can be linked with ash. The header file bltin.h |
300 | takes care of most of the differences between the ash and the |
301 | stand-alone environment. The user should call the main routine |
302 | "main", and #define main to be the name of the routine to use |
303 | when the program is linked into ash. This #define should appear |
304 | before bltin.h is included; bltin.h will #undef main if the pro- |
305 | gram is to be compiled stand-alone. |
306 | |
307 | CD.C: This file defines the cd and pwd builtins. The pwd com- |
308 | mand runs /bin/pwd the first time it is invoked (unless the user |
309 | has already done a cd to an absolute pathname), but then |
310 | remembers the current directory and updates it when the cd com- |
311 | mand is run, so subsequent pwd commands run very fast. The main |
312 | complication in the cd command is in the docd command, which |
313 | resolves symbolic links into actual names and informs the user |
314 | where the user ended up if he crossed a symbolic link. |
315 | |
316 | SIGNALS: Trap.c implements the trap command. The routine set- |
317 | signal figures out what action should be taken when a signal is |
318 | received and invokes the signal system call to set the signal ac- |
319 | tion appropriately. When a signal that a user has set a trap for |
320 | is caught, the routine "onsig" sets a flag. The routine dotrap |
321 | is called at appropriate points to actually handle the signal. |
322 | When an interrupt is caught and no trap has been set for that |
323 | signal, the routine "onint" in error.c is called. |
324 | |
325 | OUTPUT: Ash uses it's own output routines. There are three out- |
326 | put structures allocated. "Output" represents the standard out- |
327 | put, "errout" the standard error, and "memout" contains output |
328 | which is to be stored in memory. This last is used when a buil- |
329 | tin command appears in backquotes, to allow its output to be col- |
330 | lected without doing any I/O through the UNIX operating system. |
331 | The variables out1 and out2 normally point to output and errout, |
332 | respectively, but they are set to point to memout when appropri- |
333 | ate inside backquotes. |
334 | |
335 | INPUT: The basic input routine is pgetc, which reads from the |
336 | current input file. There is a stack of input files; the current |
337 | input file is the top file on this stack. The code allows the |
338 | input to come from a string rather than a file. (This is for the |
339 | -c option and the "." and eval builtin commands.) The global |
340 | variable plinno is saved and restored when files are pushed and |
341 | popped from the stack. The parser routines store the number of |
342 | the current line in this variable. |
343 | |
344 | DEBUGGING: If DEBUG is defined in shell.h, then the shell will |
345 | write debugging information to the file $HOME/trace. Most of |
346 | this is done using the TRACE macro, which takes a set of printf |
347 | arguments inside two sets of parenthesis. Example: |
348 | "TRACE(("n=%d0, n))". The double parenthesis are necessary be- |
349 | cause the preprocessor can't handle functions with a variable |
350 | number of arguments. Defining DEBUG also causes the shell to |
351 | generate a core dump if it is sent a quit signal. The tracing |
352 | code is in show.c. |