Stopwords for programming languages (for, while, print,...)

Question

I am trying to do some analysis about user behaviour when typing (keystroke biometrics). Ideally, it will include traits extracted when people are writing code. Although not technically Natural Language, code also has some structured characteristics as language and I wanted to leverage that. I was wondering if there has been some research about performing language analysis focusing on programming languages instead of traditional spoken languages.

Mainly, I am interested in having a comprehensive list of stopwords for as many languages as possible. For example, stopwords will include: for, while, return, break, string, if, else, and so on. Although it would be nice to have them separated by languages, I wouldn't mind a list comprising several languages.

I know this could be done for example by getting some sample code and retrieving the most frequent terms, but I also wanted to know if there has been some research towards this direction.

Any ideas, papers, methods would be welcome.

Thanks!

Answer 1

Stop words are common words in a language and they are usually removed when there appearance is not indicative to the analysis goal.

Suppose our goal is to text mine and find out if a given text is about sports or politics. If we use a bag of words, "not" is probably a stop word we should remove since it is probably not indicative of either categories.

On the other hand, if we would like to differ between "This article will not discuss politics" and "this article will discuss politics", we must not remove "not".

The examples you gave look like programming languages reserved words. In most languages you can find this documentation and skip the learning phase. Looking at the frequent tokens will probably also give you the reserved words (and probably plenty of English stop words that will appear in the remarks).

However, before utilising such a list, be sure that neglecting them will serve your goal. In case that typing them doesn't differ from typing other words, you might better leave them. This way you will have more data the represent better the user behaviour.

Answer 2

Here is an overview of reserved words, which dal mentioned.

Answer 3

About what you are talking about for the programming languages the right definition is reserved words. A reserved word is a word that cannot be used as an identifier, such as the name of a variable, function, or label, so it is "reserved from use". This is a syntactic definition and it is related to the compiling procedure of the source code of a program.

Stop words are not the same as reserved words. Stop words are words which are filtered out before or after processing of natural language data. Stop words usually are the most common words in a language and there is no single universal list of stop words for each different language. Stop words are related to the natural language (text) and not to the source code in any programming laguage.