Questions tagged [apache-pig]

Apache Pig is a platform for analyzing large data sets that consists of a high-level language for expressing data analysis programs, coupled with infrastructure for evaluating these programs. The salient property of Pig programs is that their structure is amenable to substantial parallelization which enables them to handle very large data sets.

Filter by
Sorted by
Tagged with
0
votes
1answer
586 views

Convert date into number - Apache PIG

Imagine that I've a field called date in this format: "yyyy-mm-dd" and I want to convert to number like "yyymmdd". For that I'm trying to use this: ...
2
votes
2answers
526 views

unable to parse XML in pig

I have a XML file has this structure (not exactly a tree though) ...
1
vote
0answers
165 views

Pig is not able to read the complete data

I am trying to load a huge dataset of around 3.4 TB with approximately 1.4 million files in Pig on Amazon EMR. The operations on the data are simple (JOIN and STORE), but the data is not getting ...
0
votes
1answer
2k views

Extract company names/job titles from free text

I have a complete Hadoop platform with HDFS, MR, Hive, PIG, Hbase, etc., Python, R, Java. All data sets have a large size. The data set A, describing the jobs of people working in a company, is ...
2
votes
1answer
1k views

Error when using MAX in Apache Pig (Hadoop)

I am trying to calculate maximum values for different groups in a relation in Pig. The relation has three columns patientid, featureid and featurevalue (all int). I group the relation based on ...
1
vote
0answers
202 views

Hadoop/Pig Aggregate Data

I am working on a project with two data sets. A time vs. speed data set (let's call it traffic), and a time vs. weather data set (called weather). I am looking to find a correlation between these ...
2
votes
0answers
803 views

Pig Rank function not generating rank in output

I am facing this bizarre issue while using Apache Pig rank utility. I am executing the following code: ...