I am even unsure if this is the correct category. I have very minimal programming knowledge. I am curious how can I get started for a project I have in mind. I want to upload a file to a webcrawler and use the data + constraint to generate an output.

Here is a hypothetical example. Suppose I have a source file of common names of fishes. And I want to find a fact about animal behavior pertaining to each fish. Would a webcrawler like Octoparse be useful?

For instance, suppose I have a list of some fishes on notepad: betta, common goldfish, common shark, seabass etc. Now I want to upload the names and have a constraint such as the fish is "red" and I need to generate just the precise location of each of them.

Thus I understand the plan is:

  1. upload a source text
  2. assign the match
  3. find the data of each field
  4. format and pare
  5. generate output file
  6. sip coffee and enjoy

Since, my skills are rusty and I have very weak background would it helpful to get started in learning the relevant programming language?

Thank you.


Glad that you mentioned our product: Octoparse. Octoparse is just a web scraping tool, but to my understanding, you require more than web scraping for your project. Octoparse might be able to with your project:

If you could find a website that contains all the information you need, you can use Octoparse to input the text (fish name) and then extract the relevant information. If the information you need from the website could be identify from its source code, you could extract them into different column. If not, then you going to need to clean it out.

The feature you may need to apply in Octoparse is the multiple texts input: https://www.octoparse.com/tutorial-7/text-input#div2


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.