Joao Pedrosa's Blog

Continuing the work on NetBeans support for my custom Ruby environment

Wed, 23 Jul 2008 00:00:00 -0000

For days I've been working with NetBeans. First I created the Ruby Tenjin support as mentioned in the previous article. Then I tunned it a bit to work a little better. The next day I created a new module whose only purpose thus far has been to add other (5) file extensions I normally use with Ruby, to the mime type handling of Ruby. After that, it was time to create a theme that mimics the one I currently use in my custom text editor for Ruby programming.

Speaking of which, a few weeks ago I had already created a theme for NetBeans to use the colors I wanted, but it was a hardcoded installation routine. But I saved it for later.

I didn't succeed with installing the theme I had created anymore as NetBeans seemed to be changed someway after I created the theme, considering that I'm using the latest development versions of NetBeans. Not to mention that it was a shameful way of installing it.

I kept digging trying to find an approach for creating/installing the theme. Therefore, soon I tried reusing one of the existing themes as a model for my own. There's a thorough theme called Aloha that tries to mimic the colors of the famous TextMate editor (on the Mac), and I used it for my own. At first I wanted one theme with the background color being standard white, as I figured the changes would be less this way. But as Aloha has an almost black background, it also could help me with finding the things I needed to change due to the larger differences.

In the process, I removed many plugins from NetBeans, though I could have gone further. I kept 18, of which 2 were mine. With the new theme plugin I was developing, it would reach 19. I did this because I would need to restart NetBeans several times to test for my changes in the theme, and it can be a painful experience to do this on and on, with plugins trying to do their jobs in the process (like the Task list). I found it enjoyable to keep just the plugins I really needed. Pretty cool this modular approach of NetBeans.

So, these are the plugins I've already created for NetBeans:

Ruby Tenjin
Ruby file extensions
Ruby theme called Vermelhou. :-)

Why Vermelhou? Because this term is quite uncommon and probably inexistent in the dictionaries, but it's used in a music that's about a carnival group that has vermelho or "red" as its color and this music really uses "red-ish" terms, which have everything to do with Ruby as well, right?

Lyrics: http://www.lyricsmode.com/lyrics/f/fafa_de_belem/vermelho.html
Music - Fafá de Belém - Vermelho: http://youtube.com/watch?v=2ccuuPASJxU

I plan on creating more support in NetBeans for the way I work.

My delving into writing a NetBeans module

Sun, 20 Jul 2008 00:00:00 -0000

So, NetBeans is this IDE many folks have tried or read about, but haven't really felt the peer pressure to use. See some of the new coming features for instance:

http://blogs.sun.com/tor/entry/multilingual_netbeans

It's written in Java and is very modularized, having a decade of development history already. It has a very active development team, and as it's open source, I could see the guys committing their stuff every day in large-enough quantities: dewd@rubynho:~/in_motion/netbeans/hg/main$ hg log --limit 1000 | grep 'Thu Jul 17' | wc
258 1806 11352

Some of these commits are "meta-commits" of things like "automatic merging" as the development uses hg (Mercurial is a distributed VCS). So the "real" commits made locally or remotely on 'Thu Jul 17' should have been below this number (258).

The module I wanted for it was support for another kind of template format. Instead of using the more standard "ERuby" and its <%= 'Hello' %> syntax, I wanted to use another format that I've come to love and use called Tenjin, that features a more XML-ish, Ruby-ish, supporting easier escaping of HTML content, which is faster even by being quite simple and by using Ruby itself for doing most of the job (so no extra C module or some such).

Tenjin uses for instance <?rb ?> (Ruby block) or #{} (expression) or ${} (escaped expression).

As NetBeans supports ERuby (RHTML) already, the idea was to copy this module into a new one and to modify it for Tenjin.

Thus it started, my fighting with Java and with NetBeans, having left the Java-land for so long and knowing zilch about NetBeans extensions development.

First the pros: these Java IDEs are advanced enough to help one with finding the obvious errors even before compiling the code. Also, NetBeans helped me handle some of the "bureaucracy" of setting dependencies for my module a good deal, once I got to know it a bit more. Plus, the foundations that have been created for NetBeans like a new Lexer with support for embedding languages and easier programming have made things quite enjoyable indeed. More, NetBeans is quite easy to compile (just increase the Heap memory available to the Ant builder; Ant being well integrated makes new modules developments well supported).

Now the cons:

Java already has some particularities of "safe programming" like "final" methods, classes, "private" ones, "protected" others... With unpublished APIs, NetBeans modules go a little further than this and have "friend dependencies" where the modules can tell which other modules can use their unpublished APIs (also APIs under development). To make a long story short, this is boring!
During the setting of my new module, for a class, I couldn't get past the "I'm overriding these two methods already" and the editor saying both that I wasn't overriding any supertype's methods and when I removed the '@override' command it offered to override them. It was copied/pasted code and I checked for the methods, they were there, they were protected, they weren't particularly available to "non-friend dependencies", but there was a workaround for using "non-friend dependencies". I removed this class after hours and hours of looking at it, digging stuff, and so on. It was a "RubyTenjinOption" one or something, so it was not crucial at this point.
Whenever I use the unpublished APIs in my module, I have to create a version-specific dependency to these other modules, meaning that just a bump in a number from version 1.86 to perhaps 1.87 could break my module on future installations of the IDE.
Somehow I couldn't get a unit test going for the RubyTenjinLexer I had to create. As it was a copy/paste from the ERuby one, and it was quite simple by reusing a generalized testing function (something "dump" was its "job"), I just needed to get the dependencies set, but couldn't for one of them (which went into the other's package ".test" subpackage).

Then to the rounding of this short article. :-)

When it came time to figuring out the "nextToken" method I needed to edit to convert from ERuby to Tenjin, which was a token scanner which got a token at a time, I looked and looked at the method, trying to figure out the external calls and what it did. I even thought of recreating some of this in pure Ruby, making it easier to test things. But even though I started the Ruby implementation of some of this, it just helped me with understanding the Java one more.

When I was comfortable enough with it, I started editing it. I got the first 3 tries slightly wrong (defining what would have been the next token to the current one). I must say that the ERuby tokenizer was slightly harder to do and I'm glad someone else did it.

This is just the first module that I've "created" for NetBeans. Hopefully as the new NetBeans APIs stabilize and get documented, things will be much easier for anyone to extend it. Nonetheless, I'm glad NetBeans exists as something like it was rather needed.

I have a collection of links should anyone want to know more about it:

http://www.deze9.com/jp/wiki/?p=netbeans

My experience thus far with ANTLR

Wed, 16 Jul 2008 00:00:00 -0000

I spent several hours playing with its development environment (ANTLRWorks) and decided to create a grammar to parse the Windows INI file format, which was simple enough to get me started. As far as I care about it though.

I don't know why, but ANTLRWorks and probably other Java Swing applications can have redrawing problem on my Linux system. It could have to do with the JVM or something: java version "1.6.0_06" Java(TM) SE Runtime Environment (build 1.6.0_06-b02) Java HotSpot(TM) Client VM (build 10.0-b22, mixed mode, sharing)

That's just the icing, as they say.

ANTLR has its own non-Regex lexer, and as I could see from some browsing, it has its legion of fans. Folks seem to like it because they can't shoot themselves in the foot too badly with it. I missed some Regex for this simple parsing, though. I found it a bit hard to make the format flexible for "any string" with it.

At times I got quite enthused when I could get the AST going, and as I was getting acquainted with ANTLR still, it was fun, overall.

Then I decided to allow for "empty" INI sections (without any key/value items), but this made things get out of control. I was able to get the empty sections on the AST, but somehow items started appearing repeated for several sections (even the empty ones) and other issues started appearing as well.

I'm not sure anymore I would want to use ANTLR for something too simple, or even for something that doesn't fit its domain too well. Sure, its error reporting is awesome, even for development. But given how hard it can be to get it going, and the expertise it might require, I'd rather use it when it was the "right tool for the right job."

No wonder such tools are not very popular as far as most developers are concerned.

On the plus side, I found it cool to start prototyping a grammar and learning ANTLR using its development environment ANTLRWorks. Perhaps this is one of its strongest selling points.

Holy Cow! It looks like Ruby-GNOME2 has started to work with Ruby 1.9 also

Thu, 22 May 2008 00:00:00 -0000

I gave it a try a few days ago and ran across what seemed like a major incompatibility with Ruby 1.9. But since Ruby 1.9 has been under constant work, some of the changes might have fixed it to work with Ruby-GNOME2 now.

Cool.

This means that I'm closer and closer to dumping Ruby 1.8 altogether.

Yay! I'm on Ruby 1.9 as far as Web Pages go. :-)

Thu, 22 May 2008 00:00:00 -0000

Since 10 minutes ago, even this server which hosts this blog and several other applications is using Ruby 1.9.

I call your attention to the handy features/changes as in the following transcription of a recent Matz presentation at Google:

Ruby 1.9 -- talk by "Matz" Matsumoto 2/22/08 - http://holtsblog.blogspot.com/2008/02/ruby-19-talk-by-matz-matsumoto-22208.html

I've found that the memory usage in Ruby 1.9 has been quite a deal more stabilized than in the Ruby 1.8 MRI version, as mentioned by Matz himself. Now Ruby 1.9 uses a little less memory and is more constant in its allocation of the memory, which can mean that the memory should stay at a minimum more often.

Also, while the speed improvements are not revolutionary in terms of application performance given the all the inefficiencies in the code and so on, Ruby 1.9 has given me 20%, 30% or more of improvements depending on the "benchmarks" (which have been informal in my tests).

I use the following libraries with Ruby 1.9 which are working for me already:

Tenjin
Thin
Ruby-pg (new postgres module compiled from trunk as of now, but it could work with an available gem)

I still have more work to do with making use of the PDF-Writer one, which doesn't seem to be supported in Ruby 1.9 yet.

I happened to come across some kind of incompatibility in file reading, but it was just once in serving the Prototype JS file which doesn't have a spare line at the end, and for some reason it caused my Web Framework to truncate it. I just added one spare line at the end of it and then it worked. And when I tried to create a reproducible example, I couldn't.

Why Ruby 1.9? I was bored with the state of the Ruby 1.8. For a moment a flirted with JRuby, and then decided that I like Ruby 1.9's improvements more. And if it means I'm "stuck" with Linux systems, given Windows is such an issue for Ruby support, so be it. This goes as far as Web Pages are concerned, though. :P

Somebody asked over at JoS: "Are other professions this self-involved?"

Fri, 08 Feb 2008 00:00:00 -0000

The thread is at http://discuss.joelonsoftware.com/default.asp?joel.3.592501.5

Here's my answer:

Some might say IT is about working with logic. I'd say IT involves a great deal of reasoning. The reasoning involves communication, reflection, experimentation... IT can both produce a lot of content and consume a lot of content, in many forms, even comics. :-) IT kind of requires a certain level of interest from the participants, and some intelligence helps people in maintaining the interest for longer periods which makes the IT experience, be it in blogging, working, discovery... more exciting to the participants. IT is a new age kind of industry. I was watching today on the National Geographic channel the evolution of the communication up to the discovery of the optical fiber, which is the basis of the Internet. This is all so new, and yet, our lives are so short. :-)

IT can be more recursive than other industries. Whatever that means. :P

IT is renewing itself all the time. IT knowledge can become a little obsolete fast, but the tools of the trade can lose their prestige much faster than the knowledge of course. At the same time, IT requires knowledge from its participants and due to the evolution of things, it falls short of teaching people beyond some minimum knowledge. After school, people start specializing for the job at hand. The specialization each one does is just one in a big universe of specializations. This creates some tunnel vision. And yet, each job can vary greatly from one another, for the same person. Many jobs are not stable.

What other industry is like IT? None.

That's why analogies fall short when comparing apples and oranges of professions and industries.

Rubinius: With great power comes great responsibility.

Sun, 03 Feb 2008 00:00:00 -0000

Somebody else was wondering about Rubinius and MRI and their future at:

What will Matz do? - http://matt.blogs.it/entries/00002788.html

The question was: "Could he make the switch and adopt Rubinius as the new MVM?"

With great power comes great responsibility.

Maintaining a development tool, like a programming language with its core libraries and even other important libraries or standard libraries, can be quite a challenge. Add to it conferences, books, community, support, decisions and more decisions, it's not a thing for the weak.

Matz and MRI have proven themselves up till now, even though there are barriers like actual language (Japanese), geographic (Japan), and even cultural, that impose some extra difficulty.

It's my understanding that some things in life can be quite hard to understand, unless you have done them yourself. It's easy to judge and to want things to improve. It's much harder to bite the bullet and to do the right things for years, even decades, with little recognition and other kinds of feedbacks.

Rubinius is just starting out. It needs more than just a good philosophy, it needs much work, probably for years to come. Think decades even! On top of that, it shouldn't screw with those guys who made Ruby what it has become. Ruby is not just a Web development platform, even though it works great for it. Ruby was a way of creating higher level programs than C allowed. Ruby was about trying to free developers of much of the low level of C programming.

Rubinius is in its very early days. When users can use Rubinius on Linux, Windows, Cygwin, MacOSX, FreeBSD, and more, it will be taken more serious as a replacement for MRI. Does Rubinius want great power? Then when tens of thousands of users start using it, Rubinius will have to deal with great responsibility as well.

For the time being, I want to see Rubinius at least working on as many platforms as it can. Rubinius should not be just a "PHP killer" if it wants to be a general purpose tool and worth of displacing MRI as the main Ruby implementation (outside of things like Java and .NET, that is).

In general, MRI has a great lead. Rubinius developers need motivation to close the gap, but they will need time as well.

Why functional programming matters? Asks Haskell's mastermind

Sat, 26 Jan 2008 00:00:00 -0000

And you can answer it at:

http://groups.google.com/group/fa.haskell/browse_thread/thread/40221a24b1f31422/9e5cd81e0bfa89d0?lnk=raot#9e5cd81e0bfa89d0

The answers are pretty uninspiring to me, all in all!

Meanwhile, here's the review done by someone else on a practical Ruby book:

Practical Ruby Projects - http://on-ruby.blogspot.com/2008/01/practical-ruby-projects-review.html

Much more inspiring, I'd say. :-)

If you are up to it, see the videos of Dave Thomas and Kent Beck on InfoQ also:

Pragmatic Dave on Passion, Skill and 'Having A Blast' - http://www.infoq.com/interviews/dave-thomas-agile-passion
Kent Beck on Implementation Patterns - http://www.infoq.com/interviews/beck-implementation-patterns

Dave Thomas is over fifty! He has done nicely so far and passed the baton to a bunch of younger folks by means of training, teaching, supporting with software, documenting... But he is not a "Ruby guy". :-) He mentions that he feels a little nervous when folks mention that they are like a "Java guy". Programming languages come and go, the job markets come and go, and the tasks might demand different approaches. Being just a "very focused on a technology kind of guy" can be detrimental to the objectives and opportunities. And things change, rest assured. :-)

Lambda the Ultimate has a fun discussion including programming languages:

Prediction for 2008 - http://lambda-the-ultimate.org/node/2600

Come what may, I say. :-)

I didn't know parsing could be such a hard job :-)

Thu, 24 Jan 2008 00:00:00 -0000

I was trying to use Treetop to parse my Wiki syntax, and it made me realize how hard such a task could be. I think this particular syntax does not suit Treetop very well.

In the process I have just grown a little tired of trying to use Treetop to do any parsing job I could have. It's not always an easy task, but it somehow felt good when I used it to parse the subset of SQL I care about. Albeit now I see that things like possible left recursiveness, too flexible syntaxes (like "

<<EOS [...] EOS or <<any [...] any

"), and other complexities can make it less fun.

For now, I am going to keep my hand-crafted parsers as they are tested in the field, and departing from them may cause incompatibilities and possibly other problems for me. :-)

Why, oh why? :-)

Having taken a look on rparsec and Ragel, I think I'm gonna resume my work on Treetop

Mon, 21 Jan 2008 00:00:00 -0000

rparsec seemed too complex for me to grasp at a glance. Ragel seems by design less adequate than Treetop for recursion.

They are all cool technologies. Each has some strong points, and in the hands of skilled people can be made to sing and dance all night long. :-)

But Treetop just feels too good to be true, and if I can make it to work, the simplifications will be worth the trouble. And in the end, I know I'll reach for Treetop in more occasions, as it's so easy to use. If I used something more complex, it would be used less often. For example, while starting a new parser manually is easy, the continued development is rather troublesome. Treetop on the other hand works at a higher level, giving an example to the "keep it simple, sir". :-)

If Treetop despite being a little slower can help me with correctness, it's worth the trouble, definitely!

Thanks for Treetop, Nathan Sobo! He is the author.