What's Relevant in the Ruby Universe?

Omniref says code and documentation need a full Ruby-wide graph, not gem-by-gem views. To build it, the system parses classes, methods, and modules across every gem, stores the relationships in a large normalized table, and uses distributed workers to resolve dependencies and avoid broken links. It then deserializes only the relevant part of that graph for a gem to check documentation against symbol references in context.

Reading notes

#

• The update is a near complete rewrite of the code analysis pipeline.
• RDoc and YARD only look within one gem, so Omniref builds the full object graph across Ruby code.
• The graph is built by parsing classes, methods, modules, and their relationships for every Ruby gem.
• The parsed data is serialized into a normalized, compressed, clustered 25GB table.
• The work takes a few thousand machine hours, so Omniref uses a cluster.
• Shared dependency resolution adds coordination problems across workers, including requeuing, deadlocks, races, and circular dependencies.
• For a given gem, Omniref deserializes the relevant part of the graph in memory and checks whether documentation words are symbol references in local context.
• The team describes this as a limited implementation of a distributed Ruby compiler and linker.
• Early signals from the full web of Ruby code are being used to improve search quality.
• Examples mentioned include active support being used more than rails, activerecord 3.0 remaining the most popular dependency, and ruby fitting data transformation work well.
• The lists of included modules, inherited classes, and overridden methods are dominated by web staples, Object, Exception, and dynamic-method features like method_missing.
• The new system also shows a class’s full ancestry in headings and inlines relevant documentation from other libraries into a single page.