Language-aware (semantic) symbol search

[jglick]

I see that https://cs.github.com/about/faq#languages supports “symbol extraction”. As far as I can tell, that means it does a lexical analysis of sources to find symbols (identifiers). But that is nowhere near adequate. Suppose I have an organization with hundreds of repositories in which one repository includes a src/main/java/org/mine/api/Api.java

package org.mine.api;
public interface Api {
    void doSomething();
    void close(boolean force);
}

and another repository with src/main/java/org/mine/impl/Impl.java

package org.mine.impl;
import org.mine.api.Api;
public class Impl implements Api, AutoCloseable {
    @Override public void doSomething() {…}
    @Override public void close() {
        close(false);
    }
    @Override public void close(boolean force) {…}
}

If I am exploring Api.java, searching for the symbol close is useless—I will get thousands of hits across the org, most of them totally unrelated, including one unrelated hit right in Impl.java. I want to know who is calling and/or implementing org.mine.api.Api.close(boolean) specifically. In a statically-typed language, it is possible to know the answer for sure (modulo weird build system tricks, reflection, etc.). Sorcerer did this years ago for Java; you need to run an actual compiler front-end up to the symbol resolution phase (in a very lax mode suppressing errors), so that you can determine that a given identifier actually refers to a specific overload of a method in a specific type, or whatever the case may be.

[dcreager]

Hi @jglick, thanks for reaching out! This is definitely on our radar. We have just (as of 30 minutes ago) announced stack graphs, which is our framework for extracting this kind of more detailed symbol information without requiring a build. It's currently only enabled for Python, but we are hard at work adding stack graph support for other languages, as well.

[jglick]

Great news! I wish you best of luck interpreting Java’s symbol resolution rules without reusing an existing battle-hardened compiler (javac or JDT). The “hard” problem mentioned at the end of your talk (Java 5 syntax) actually seems fairly straightforward compared to the issues I have seen in practice. For the foreseeable future I expect search results should be treated as a best effort, suitable for casual exploration, with other tools used to answer definitive usage questions such as whether it is safe to delete a deprecated method.