Skip to content

Issues in lab 5: hdfs_2gram_data_load #75

Description

@samkabue
  • the link '(see Hadoop v1 Installation)' is broken (it points to https://github.com/MIDS-scaling-up/coursework/blob/master/week5/hw/version_1/README.md) which is a 404
  • after setup, /usr/local/hadoop/contrib/streaming/hadoop-streaming-1.2.1.jar (in Part 2: Load data) is not valid---maybe this should be updated to /usr/local/hadoop/share/hadoop/tools/lib/hadoop-streaming-2.7.1.jar?
  • when I fix the previous issue with the suggested workaround, and then do 'Part 2: Load data', I get a lot of errors like the following:
  Container [pid=32025,containerID=container_1443213849013_0005_01_000002] is running beyond virtual memory limits. Current usage: 159.9 MB of 1 GB physical memory used; 2.2 GB of 2.1 GB virtual memory used. Killing container.

  ...

  ERROR streaming.StreamJob: Job not successful!
  Streaming Command Failed!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions