Difference between revisions of "Apache Hadoop"
Jump to navigation
Jump to search
Tags: Mobile web edit, Mobile edit |
|||
(6 intermediate revisions by the same user not shown) | |||
Line 1: | Line 1: | ||
− | |||
[[wikipedia:Hadoop]] ([[2006]]) | [[wikipedia:Hadoop]] ([[2006]]) | ||
* [[HDFS]] | * [[HDFS]] | ||
* [[MapReduce]] | * [[MapReduce]] | ||
+ | * <code>[[brew install hadoop]]</code> | ||
+ | [[hadoop fs]] | ||
− | There are currently several monitoring platforms to track HDFS performance, including [[Hortonworks]], [[Cloudera]], and Datadog. | + | There are currently several monitoring platforms to track HDFS performance, including [[Hortonworks]], [[Cloudera]], and [[Datadog]]. |
+ | == Example == | ||
+ | <pre> | ||
+ | hadoop | ||
+ | Usage: hadoop [OPTIONS] SUBCOMMAND [SUBCOMMAND OPTIONS] | ||
+ | or hadoop [OPTIONS] CLASSNAME [CLASSNAME OPTIONS] | ||
+ | where CLASSNAME is a user-provided Java class | ||
+ | |||
+ | OPTIONS is none or any of: | ||
+ | |||
+ | --config dir Hadoop config directory | ||
+ | --debug turn on shell script debug mode | ||
+ | --help usage information | ||
+ | buildpaths attempt to add class files from | ||
+ | build tree | ||
+ | hostnames list[,of,host,names] hosts to use in worker mode | ||
+ | hosts filename list of hosts to use in worker | ||
+ | mode | ||
+ | loglevel level set the log4j level for this | ||
+ | command | ||
+ | workers turn on worker mode | ||
+ | |||
+ | SUBCOMMAND is one of: | ||
+ | |||
+ | |||
+ | Admin Commands: | ||
+ | |||
+ | daemonlog get/set the log level for each daemon | ||
+ | |||
+ | Client Commands: | ||
+ | |||
+ | archive create a Hadoop archive | ||
+ | checknative check native Hadoop and compression libraries | ||
+ | availability | ||
+ | classpath prints the class path needed to get the Hadoop jar | ||
+ | and the required libraries | ||
+ | conftest validate configuration XML files | ||
+ | credential interact with credential providers | ||
+ | distch distributed metadata changer | ||
+ | distcp copy file or directories recursively | ||
+ | dtutil operations related to delegation tokens | ||
+ | envvars display computed Hadoop environment variables | ||
+ | fedbalance balance data between sub-clusters | ||
+ | fs run a generic filesystem user client | ||
+ | gridmix submit a mix of synthetic job, modeling a profiled | ||
+ | from production load | ||
+ | jar <jar> run a jar file. NOTE: please use "yarn jar" to | ||
+ | launch YARN applications, not this command. | ||
+ | jnipath prints the java.library.path | ||
+ | kdiag Diagnose Kerberos Problems | ||
+ | kerbname show auth_to_local principal conversion | ||
+ | key manage keys via the KeyProvider | ||
+ | rbfbalance move directories and files across router-based | ||
+ | federation namespaces | ||
+ | rumenfolder scale a rumen input trace | ||
+ | rumentrace convert logs into a rumen trace | ||
+ | s3guard S3 Commands | ||
+ | version print the version | ||
+ | |||
+ | Daemon Commands: | ||
+ | |||
+ | kms run KMS, the Key Management Server | ||
+ | registrydns run the registry DNS server | ||
+ | |||
+ | SUBCOMMAND may print help when invoked w/o parameters or with -h. | ||
+ | </pre> | ||
+ | |||
+ | |||
+ | |||
+ | == Related terms == | ||
+ | * [[Apache Hive]] (2010) | ||
+ | * [[Apache Zookeeper]] | ||
== See also == | == See also == | ||
Line 13: | Line 85: | ||
* {{Cloudera}} | * {{Cloudera}} | ||
* {{Apache}} | * {{Apache}} | ||
− | |||
[[Category:Computing]] | [[Category:Computing]] | ||
+ | [[Category:Apache]] |
Latest revision as of 18:29, 9 April 2024
hadoop fs
There are currently several monitoring platforms to track HDFS performance, including Hortonworks, Cloudera, and Datadog.
Example[edit]
hadoop Usage: hadoop [OPTIONS] SUBCOMMAND [SUBCOMMAND OPTIONS] or hadoop [OPTIONS] CLASSNAME [CLASSNAME OPTIONS] where CLASSNAME is a user-provided Java class OPTIONS is none or any of: --config dir Hadoop config directory --debug turn on shell script debug mode --help usage information buildpaths attempt to add class files from build tree hostnames list[,of,host,names] hosts to use in worker mode hosts filename list of hosts to use in worker mode loglevel level set the log4j level for this command workers turn on worker mode SUBCOMMAND is one of: Admin Commands: daemonlog get/set the log level for each daemon Client Commands: archive create a Hadoop archive checknative check native Hadoop and compression libraries availability classpath prints the class path needed to get the Hadoop jar and the required libraries conftest validate configuration XML files credential interact with credential providers distch distributed metadata changer distcp copy file or directories recursively dtutil operations related to delegation tokens envvars display computed Hadoop environment variables fedbalance balance data between sub-clusters fs run a generic filesystem user client gridmix submit a mix of synthetic job, modeling a profiled from production load jar <jar> run a jar file. NOTE: please use "yarn jar" to launch YARN applications, not this command. jnipath prints the java.library.path kdiag Diagnose Kerberos Problems kerbname show auth_to_local principal conversion key manage keys via the KeyProvider rbfbalance move directories and files across router-based federation namespaces rumenfolder scale a rumen input trace rumentrace convert logs into a rumen trace s3guard S3 Commands version print the version Daemon Commands: kms run KMS, the Key Management Server registrydns run the registry DNS server SUBCOMMAND may print help when invoked w/o parameters or with -h.
Related terms[edit]
- Apache Hive (2010)
- Apache Zookeeper
See also[edit]
- Apache Hadoop, HDFS, MapReduce
- Cloudera, Hadoop, Cloudera Data Platform (CDP)
- Apache: Apache Mesos, Apache Cassandra, Hadoop YARN, Apache CloudStack, Apache Hadoop, Apache Maven, Apache Ivy, Apache Ant, Apache Ignite, Apache Flume, Apache HTTP server, Apache Archiva, Apache Karaf, Apache Flink, Apache Beam, Apache Solr, Apache JMeter, Apache Groovy, Apache NiFi,
RewriteEngine
, Apache CouchDB, Apache Parquet, Apache Chainsaw, Apache Spark, Apache Lucene, MXNet
Advertising: