I'm almost 40.
Four months ago I resigned from work to rest from stressing work and change my carerr direction. It took me some time to understand what I want to do and only then I started to learn NLP and machine learning through Coursera and Kaggle. To learn new skills and have a minimal portfolio to show during future job interviews, I've started to work on a small NLP project. It involves multiple problems like crawling data from the Web, cleaning them, extracting structured data from unstructured text, linking objects based on document similarity, stemming, POS tagging, lemma disambiguation, etc.
However, I feel I'm getting lost and often going into wrong directions. Actually, it starts to look like doing my PhD in the past, reinventing too many wheels. For instance, last week I've spent on porting existing stemmer for my language from Java to Python just to realize a few things. First, that less ideal but simpler solution (Python wrapper) would be enough for my goals and definitely faster to write. Second, I've learned more about software engineering (difference between Java and Python, calling Java from Python) rather than about NLP challenges of the problem (what are the current approaches for stemming, how to train trie-based stemmers, etc.). Third, I've found that the approach I used might be a bit outdated.
I feel lost because of many unknowns:
- I don't know if the project that I took makes sense for my future career?
- Isn't focusing on my language in my NLP project too constraining? NLP tooling for my language is quite limited when compared to English. Who knows what are uknown uknowns?
In the end I think it boils down to lack feedback on my progress and actual skills I have vs. what is needed. The only feedback I've received so far was from one job interview where I've learned that the approach I'm using for my project might be outdated and not necessarily an industry standard. I've looked at the work of a group of students contributing to SpaCy, NLP framework in Python. They have a project supervised by a univeristy teacher or principal, the project requirements have been defined by an external company and they have a chance to work together.
How to get such a feedback when you're no longer a student? Internship? Mentoring? How do you learn alone such things?