GSoC Final Summary and Thoughts

Now, some numbers:

I added 707 test cases in total. I got this number by adding up the results of the commit messages containing /\d+ test/ (adding a few known misses and only getting the ones that had a gsoc_spectest).

I made 105 commits to the Pugs repo. Most of these had to do with tests, but probably 10 dealt with the p6eval IRC bot. Most of my commit messages were two lines long.

I have a karma of 166 in #perl6 and 18 in #parrot. Useless information, but most numbers are useless. :)


This image gives some more useful numbers. This is the result of the spectest_regression graph (courtesy of pmichaud++ and moritz++), from May 26th to August 18th. Those dates just happen to be the bounds of my SoC project. I'm only responsible for a few hundred tests in there, but it shows some of the progress that is related to my project. rakudo.de has a graph, updated most days.

I've really enjoyed working with everyone in the Perl 6 community this summer, and I've learned a lot. It was a fun experience, and I hope that I brought Christmas significantly closer. I'd like to say thanks to Moritz, Larry Wall, Particle, and pmichaud for helping mentor, being active in my #perl6-soc meetings, and putting up with me in general. I'd also like to thank the Perl Foundation and @Larry for accepting my project.

Now, school begins in a few days, and I have a few other projects to tend to. I will still reside in #perl6, and I hope to continue to contribute when I can.

GSoC 2008 Week 12: S06 (Subroutines)

This week concludes my Summer of Code project. I spent this week on S06, subroutines. This section was missing quite a few tests that were easy to find with ack. Some of these tests found omissions in STD.pm's parser, which were quickly fixed by TimToady++.

I added 59 tests this week, and moved around 75 into t/spec/ after verifying that they are correct and fudging (where it made sense).

I'm going look through svn log to get my totals and post some closing thoughts shortly.

GSoC 2008 Week 10: S05 (Regexes)

I did more S05 work this week. My work is cut a bit short due to vacation beginning tomorrow for me. This means that S06 is pushed to the final week, and I won't be doing any bonus work.

According to my svn logs, I added 44 regex tests this week. Rakudo's IO was improved by Jonathan++ on one of his Rakudo days, so it shouldn't be too difficult to merge in the larger test cases. I'll investigate that more when I return from vacation. My testing caught several bugs in STD.pm. Of course, it caught several bugs in my tests. I've made a habit of testing my tests with it to catch dumb syntax errors, since STD.pm parses almost all of t/spec (and a lot of t/, too, with some POD and other fixes from TimToady++).

I attempted to get STD.pm working within p6eval on #perl6, but it was mysteriously not working. I think my update script broke the file permissions or something, so that's something more to investigate.

I may have some time to work on tests while traveling this week, but I don't count on it. If so, I'll add some tests. If not, this is why we add spare time to the schedule. See you in a week!

GSoC 2008 Week 9 Report: S04 and S05 (Regexes and Rules)

This week, I finished up work on S04 and began working on S05, regexes and rules. I also added a handful of tests in S09 (data structures), which still has quite a few easy areas that I may revisit.

S05 is probably the most complicated synopsis that I've encountered so far. It requires a very large amount of testing. There are 3000+ tests written for regexes outside of spec, and I've been moving them into spec while verifying that they are still correct. In addition, there are also around a thousand Parser Grammar Engine tests in the Parrot repository. PGE implements Perl 6 rules, so these tests can be easily reused. There is bound to be some overlap, but simplifying them can be done when verifying that they are up to date.

In reorganization news, I am continuing to move tests from t/ into t/spec, and we're right at 11000 tests that aren't in spec by my script that does a rough count. We have around 6000 tests in spec now by the same rough count.

One issue that I am encountering is organizing the tests. I'm trying to organize them by heading when possible and reference the spec and other tests files whenever possible.

I finished 78 tests this week. I didn't expect this week to be as productive, but somehow it worked out. Next week is more S05 testing, and there's a lot more there, especially in the reorganization area. Week after that is vacation, then I finish up the summer with S06 in the final week.

GSoC 2008 Week 8 Report: S04 (Blocks and Statements)

I worked on S04 (Blocks and Statements) last week. It was not nearly as productive as I hoped, even though I knew that it would be disrupted by other obligations. I only managed to add a few tests and fudge a few more files.

This week was originally supposed to be more work on S04 and some work on S05. However, I want to return to S09 for a bit this week and grab some more of the low-hanging fruit there. So my plan is to look through S04 more today, spend one or two days on S09, and then start work on S05, which defines regexes.

GSoC 2008 Week 7 Report: S09 (data structures)

S09 describes the data structures in Perl 6, covering topics such as low-level data types, multidimensional arrays, and user-indexed arrays. Without a doubt, this is the most undertested synopsis so far.

I wrote 98 tests on Monday alone. Admittedly, they were the same 7 cases over and over, but they needed to be tested. I then added another 27 cases for arrays of fixed sizes and types, and then another 10 tonight for basic multidimensional arrays. I could probably write another 100 or so on this synopsis, too.

My total for the week is 135, which should help for those less productive weeks. Unfortunately, I only scratched the surface for tests to write for this synopsis, so I plan to return to it if I feel unproductive during an upcoming week, or perhaps towards the end if I have spare time (yeah right ;) ).

This brings my total for the summer to 507, which puts me nearly on track for the summer. I did not do any fudging this week since I did not see any that was related to this section.

I will be missing a few days next week, but I hope that it is as productive as this one was. I will be covering S04 next week, which is Blocks and Statements. I will be doing S04 and S05 like S02/S03, which gives each of them 10.5 days instead of the usual 7.

It is very likely that I will be taking a week's vacation in the first week of August, which will eat up my slack time. This is unfortunate, since I could put it to good use (especially with S09's testing possibilities), but this is why I planned slack time.

GSoC 2008 Week 6 Report: S03 (summary of Perl 6 operators)

Week 6 is now complete. I continued worked on S03, and found quite a few range tests to add that passed in Rakudo (and of course added some that failed in Rakudo).

I only added 42 tests this week. I had two things that prevented me from getting a full 75 tests done: First, July 4th ate into my time and prevented me from working a few days. Second, the lure of fudging tests to pass in Rakudo. It's quite fun to see how many tests are in the chart the next day.

Some of my fudging effort can be seen be looking at this chart (or, to see Sunday's work, look at this one in about 12 hours). Note the significant increase this previous week. I spent about half of my time this week contributing to that effort, and I added somewhere around 400 tests to that chart.

moritz++ and pmichaud++ contributed to this effort significantly as well. In addition to just fudging tests, we converted POD, moved tests from t/ to t/spec/, and otherwise reviewed the tests.

Next week, I'll be handling S09, Data Structures. I see a lot of sections missing linked tests. I also plan to do a bit of reorganization, but not nearly as much as I did this week.

GSoC 2008 Week 5 Report: S02 and S03 (summary of Perl 6 operators)

Week 5 is over. I split this week on S02 and S03 so that each of the two would have roughly 10 days each. I'll do the same thing for S04/S05, which are also fairly large. S03 has a lot of things that are hard to search for due to the use of punctuation, but ack and the smarlinks are very helpful as always. S03 has a lot of things to test, too, including junctions, ranges, equality, and precedence of operators. I'm not sure how to search for precedence other than searching for the word 'precedence' in the test descriptions and comment.

I added 78 tests this week and did a bunch of reorganization. This puts my total tests at 330 tests for 5 weeks.

The bulk of my commits were for S03, including 28 tests that I did last night when I saw some easy tests. My reorganization work included the package tests, which were referenced in S02. I moved them to t/spec/S10-packages which should be a more obvious place to find them instead of one of the S02 directories. I ran into the problem that S10-packages wasn't a valid portion of a package name, so some of the package files ended up in t/spec/packages.

I also fixed some POD, smartlinks, and moved individual test files that I edited into a subdirectory of spec.

Next week I continue working on S03. My TASKS still has a bunch of entries for S03 that I didn't handle yet, so there should be some easy things to test yet.

GSoC 2008 Week 4 Report: S02, bits and pieces

Last week, I worked on S02, bits and pieces. As the name implies, it has a lot of everything. The bulk of my tests that I created were quoting tests. Yesterday, I worked on tests for the item, list, hash, and slice contextualizers, which ended up being 32 tests. There are still more things to test with the contextualizers, especially with captures and the slice context.

Since some of the contextualizer tests worked in Rakudo, but some failed, I learned how to fudge the ones that didn't work. Fudge is a neat tool for this, but caused me some problems when I accidentally left off a semicolon on one fudged test statement and it removed the next statement as well.

Rakudo doesn't pass that many test cases compared to Pugs and the total test count. Rakudo passed 843 test cases from the Pugs test suite at midnight last night. (pmichaud++ and others are tracking the number of passing test cases in a csv file in the Parrot repo. My 17 new test cases will be counted tomorrow since they came in after midnight). Pugs passes around 10000 and the repo holds around 18000 or 19000. I think that Rakudo will start passing many more tests in the upcoming weeks, as I have a feeling that there are just a few things that hold it back from passing a lot of tests.

I added 64 tests this week. Next week, I'll be splitting my time between S02 and S03, and the week after that I'll spend completely on S03. S03 is the summary of Perl 6 operators, and I already have a few things lined up for testing in t/TASKS.

GSoC 2008 Week 3 Report: S12, objects

This week, I handled S12, which covered objects. A majority of the existing tests existed in t/oo, and some of those tests were adapted from the tests from the CPAN module Moose. I left most of the tests alone rather than move them, since that would take a lot of time without adding many new tests. I may move them in the not so distant future.

I had some difficulties with some of the more advanced aspects of the object system that I haven't seen before. I was impressed by the 'deferring' functionality (provided by callsame, callwith, nextsame, nextwith, and ended by lastcall). This functionality was very easy to understand, but the exact order was unclear (and Larry++ said that he hasn't decided yet), which made it difficult to test. I ended up testing for an order that made sense to me, and it's easy enough to change it if it's wrong. The items that I left in t/TASKS were similar in that they were not specced enough to be easily tested.

I usually try running sample code before writing a test case. I was surprised (and impressed!) when Rakudo had something untested but already implemented.

This week was very productive in the added test department. I added 76 tests during the week. I modified only around 10 tests, which were a result of other changes in the spec. Note that my modification count does not include the tests in the files that I moved. This puts my total tests added at 188.

Next week I begin work on S02, bits and pieces. S02 mentions a lot of things in Perl 6, so it'll allow me to test pretty much anything that I see. I'll be spending all of next week on this synopsis, and splitting the week after between S02 and S03.

GSoC 2008 Week 2 Report: S29, builtin functions

Last week, I tackled S29, builtin functions. This is one of the bigger synopses for Perl 6. It defines all the methods/functions that apply to the main data types, such as Lists, Arrays, Hashes, and Strings. My approach has been to read over the synopsis, looking for issues with the existing tests or for cases that aren't tested at all. The builtins can be called as both functions and methods, so I use the terms interchangeably here.

Of course, if you look at my projected timeline, you'll notice that I did S29 a week earlier and skipped S12. I wish I could say it was on purpose, but it wasn't. I'll do S12 next week instead.

I (re)started a list of tasks for the test suite in the Pugs repo at /t/TASKS. As noted in the file, I will be watching for requests for tests. This file is for anyone looking for something to do that only requires some Perl 6 knowledge.

I ran into a few things that were problematic. The samebase function required more Unicode knowledge than I have, but I was still able to add some basic tests. A few other methods require more clarification as well. I added these methods to TASKS.

Now, the numbers. This week, I kept track of my added/modified numbers within the SVN commit logs. I added 39 tests this week, and modified 15 tests. Most of the additions were for sprintf, and I plan on continuing to add tests for it and the other things I mentioned in TASKS.

Next week, I will handle S12, objects.

GSoC 2008 Week 1 Report

I've spent a full week on my SoC project now. This week, I added tests for S01, S10, S11, and S13. Most of the tests that I added were for S13 (overloading), a few to S11 (modules), and I didn't add any for S01 (intro) or S10 (packages).

In theory, you can search SVN for commits marked [gsoc_spectest], but in practice, I misspelled it as "spec_teset" once and forgot to tag a minor one. I also did a bit of organization and added a few cases to S02 (bits and pieces), which I'll be working on during weeks 4 and 5. All my commits are viewable here: http://moritz.faui2k3.org/svn-search/search.pl?q=author%3Aauzon

When I choose the synopses for each week, I choose some shorter ones for the first week, hoping that they would be easier and allow me to work out any issues. It was nice to have some shorter synopses to work on, but the longer one (S13) was the easiest to test out of the four.

Next week, I'll be working on S29 (built-in functions). I looked at the spec yesterday for a few minutes, and I can see many areas for improvement already.

As for the numbers: I added 67 test cases in these areas, plus 6 more in S02 for a total of 73 test cases. This number was obtained with a painful series of commands, listed here so I can refer back to it if needed. I promise that it'd look better if I put it in a file and commented it. svn log -v -r20000:HEAD /home/ak9/pugs/t | perl -le'my $s = join "", <>; my @r = split /\-{60,}/, $s; %h = map {$_ => 1} grep {-e $_} map {"/home/ak9/pugs$_"} map {m~A\s+([/\-\w]+\.t)~g} grep {/\[gsoc/ && /Auzon \|/} @r; print for keys %h' | xargs perl -lne'print $1 if /plan\s+(\d+);/' | perl -le'$t = 0; while(<>) {$t += $_}; print $t'
Unfortunately, this isn't much help since this will falsely report the files I moved as counting towards my total. I'll have to keep a daily tally instead.