Docs/manual.texi

updates for the test suite
Docs/Support/test-make-manual
    abort with a message if a utility fails
    open the manual in a browser on success

Docs/Support/test-make-manual

@@ -1,13 +1,31 @@
 #!/bin/sh
 
+function die
+{
+  echo $1
+  exit 1
+}
+
 echo
 echo "|---- Running makeinfo ----|"
 makeinfo --no-split -I . manual.texi
 
+[ $? != 0 ] && die "Manual has errors - fix before you commit"
+
 echo
 echo "|---- Running texi2html ----|"
 /usr/bin/perl ./Support/texi2html -iso -number manual.texi
 
+[ $? != 0 ] && die "Manual has errors - fix before you commit"
+
+[ -z $BROWSER ] && BROWSER=netscape 
+
 echo
 echo "Please examine your modifications in \`manual.html'."
-echo
+echo "For your convenience, I will show you the manual in $BROWSER"
+echo "If you would like to use a different browser, set BROWSER enviroment\
+ variable" 
+echo "If this is a GUI browser, to get your shell prompt back, close the
+ window"
+
+$BROWSER file://`pwd`/manual_toc.html

Docs/manual.texi

@@ -817,6 +817,7 @@ MySQL Internals
 
 * MySQL threads::               MySQL threads
 * MySQL full-text search::      MySQL full-text search
+* MySQL test suite::            MySQL test suite
 
 MySQL Full-text Search
 
@@ -37823,11 +37824,17 @@ safe, you should take a look at @code{PostgreSQL}.
 @chapter MySQL Internals
 
 This chapter describes a lot of things that you need to know when
-working on the @strong{MySQL} code.
+working on the @strong{MySQL} code. If you plan to contribute to MySQL
+development, want to have access to the bleeding-edge in-between
+versions code, or just want to keep track of development, follow the
+instructions in @xref{Installing source tree}. . If you are intersted in MySQL
+internals you should also subscribe to internals@@lists.mysql.com - this is
+a relatively low traffic list, in comparison with mysql@@lists.mysql.com .
 
 @menu
 * MySQL threads::               MySQL threads
 * MySQL full-text search::      MySQL full-text search
+* MySQL test suite::            MySQL test suite
 @end menu
 
 @node MySQL threads, MySQL full-text search, MySQL internals, MySQL internals
@@ -37872,7 +37879,7 @@ DELAYED} threads.
 @cindex searching, full-text
 @cindex full-text search
 @cindex FULLTEXT
-@node MySQL full-text search,  , MySQL threads, MySQL internals
+@node MySQL full-text search,MySQL test suite, MySQL threads, MySQL internals
 @section MySQL Full-text Search
 
 Since Version 3.23.23, @strong{MySQL} has support for full-text indexing
@@ -38007,6 +38014,125 @@ There is no need to rebuild the indexes though.
 
 @end itemize
 
+@node MySQL test suite, ,MySQL full-text search, MySQL internals
+@section MySQL Test Suite
+
+Until recently, our main full-coverage test suite was based on
+proprietary customer data and for that reason was not publically available. The
+only publically available part of our testing process consisted of
+@code{crash-me} test, the Perl DBI/DBD benchmark found in @code{sql-bench}
+directory, and miscalaneous tests combined in @code{tests} directory. The lack
+of a standardazied publically available test suite made it hard for our users
+as well as developers to do regression tests on MySQL code. To address this
+problem, we have created a new test system that is  included in the source
+and binary distributions starting in version 3.23.29.
+
+The test system consist of a test language interpreter @code{mysqltest},
+@code{mysql-test-run} - a shell script to run all tests, the actual test cases
+written in a special test language, and their expected results. To run the
+test suite on your system after a build, type @code{mysql-test/mysql-test-run}
+from the source root. If you have installed a binary distribution, @code{cd}
+to the install root ( eg. @code{/usr/local/mysql} ), and do
+@code{scripts/mysql-test-run}. All tests should succeed. If they do not,
+use @code{mysqlbug} and send a bug report to @email{bugs@@lists.mysql.com}.
+Make sure to include the output of @code{mysql-test-run}, contents of all
+@code{.reject} files in @code{mysql-test/r} directory.
+
+If you have a copy of @code{mysqld} running on the machine where you want to
+run the test suite you do not have to stop it, as long as it is not using
+ports @code{9306} and @code{9307}. If one of those ports is taken, you should
+edit @code{mysql-test-run} and change the values of the master and/or slave
+port to something not used. 
+
+The current set of test cases is far from comprehensive, as we have not yet
+converted all of our private suite tests to the new format. However,
+it should already catch most obvious bugs in the SQL processing code,
+OS/library issues, and is quite thorough in testing replication. Our eventual
+goal is to have the tests cover 100% of the code. We welcome contributions to
+our test suite. You may especially want to contribute tests that examine the
+functionality critical to your system - this will ensure that all
+@strong{MySQL} releases will work well with your applications.
+
+You can use @code{mysqltest} language to write your own test cases.
+Unfortunately, we have not yet written full documentation for it - we plan to
+do this shortly. However, you can look at our current test cases and
+use them as an example. Also the following points should help you get started:
+
+@itemize
+@item
+The tests are located in @code{mysql-test/t/*.test}
+@item
+You can run one individual test case with
+@code{mysql-test/mysql-test-run test_name}
+removing @code{.test} extension from the file name
+@item
+A test case consists of @code{;} terminated statements and is similar to the
+input of @code{mysql} command line client. A statement by default is a query
+to be sent to @strong{MySQL} server, unless it is recognized as internal
+command ( eg. @code{sleep} ).
+@item
+All queries that produce results, eg @code{SELECT}, @code{SHOW},
+@code{EXPLAIN}, etc, must be preceded with @code{@@/path/to/result/file}. The
+file must contain expected results. An easy way to generate the result file
+is to run @code{mysqltest -r < t/test-case-name.test} from @code{mysql-test}
+directory, and then edit the generated result files, if needed, to adjust
+them to the expected output. In that case, be very careful about not adding or
+deleting any invisible characters - make sure to only change the text and/or
+delete lines. If you have to insert a line, make sure the fields are separated
+with a hard tab, and there is a hard tab at the end. You may want to use
+@code{od -c} to make sure your text editor has not messed anything up during
+edit. We, of course, hope that you will never have to edit the output of
+@code{mysqltest -r} as you only have to do it when you find a bug.
+@item
+To be consistent with our setup, you should put your result files in
+@code{mysql-test/r} directory and name them @code{test_name.result}. If the
+test produces more than one result, you should use @code{test_name.a.result},
+@code{test_name.b.result}, etc
+@item
+Failed test results are put in a file with the same name as the result file
+followed by @code{.reject} extenstion. If your test case is failing, you
+should do a diff on the two files. If you cannot see how they are different,
+examine both with @code{od -c} and also check their lengths.
+@item
+You can prefix a query with @code{!} if the test can continue after that query
+returns an error.
+@item
+If you are writing a replication test case, you should on the first line put
+@code{source include/master-slave.inc;} at the start of the test file. To
+switch between master and slave, use @code{connection master;}/
+@code{connection slave;}. If you need to do something on an alternate
+connection, you can do @code{connection master1;} for the master, and
+@code{connection slave1;} on the slave
+@item
+If you need to do something in a loop, you can use someting like this:
+@example
+let $1=1000;
+while ($1)
+@{
+ # do your queries here
+ dec $1;
+@}
+@end example
+@item
+To sleep between queries, use @code{sleep} command. It supports fractions of
+a second, so you can do @code{sleep 1.3;}, for example, to sleep 1.3 seconds.
+@item
+To run the slave with additional options for your test case, put them
+in the command-line format in @code{mysql-test/t/test_name-slave.opt}. For
+the master, put them in @code{mysql-test/t/test_name-master.opt}.
+@item
+If you have a question about the test suite, or have a test case to
+contribute, e-mail to
+@email{internals@@lists.mysql.com}. As the list does not accept attachemnts,
+you should ftp all the relevant files to
+@url{ftp://support.mysql.com/pub/mysql/Incoming}
+
+@end itemize
+
+
+
+
+
 @page
 @cindex environment variables, list of
 @node Environment variables, Users, MySQL internals, Top