2022-12-12 17:37:40,067 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager: Initialized queue: root.default
2022-12-12 17:37:40,067 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager: Initialized queue: root
2022-12-12 17:37:40,071 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager: Initialized root queue root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=<memory:0, vCores:0>usedCapacity=0.0, numApps=0, numContainers=0
2022-12-12 17:37:40,073 INFO org.apache.hadoop.yarn.server.resourcemanager.placement.UserGroupMappingPlacementRule: Initialized queue mappings, override: false
2022-12-12 17:37:40,073 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.WorkflowPriorityMappingsManager: Initialized workflow priority mappings, override: false
2022-12-12 17:37:40,074 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.placement.MultiNodeSortingManager: MultiNode scheduling is 'false', and configured policies are
2022-12-12 17:37:40,074 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler: Initialized CapacityScheduler with calculator=class org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator, minimumAllocation=<<memory:1024, vCores:1>>, maximumAllocation=<<memory:8192, vCores:4>>, asynchronousScheduling=false, asyncScheduleInterval=5ms,multiNodePlacementEnabled=false, assignMultipleEnabled=true, maxAssignPerHeartbeat=100, offswitchPerHeartbeatLimit=1
2022-12-12 17:37:40,081 INFO org.apache.hadoop.conf.Configuration: dynamic-resources.xml not found
2022-12-12 17:37:40,086 INFO org.apache.hadoop.yarn.server.resourcemanager.AMSProcessingChain: Initializing AMS Processing chain. Root Processor=[org.apache.hadoop.yarn.server.resourcemanager.DefaultAMSProcessor].
2022-12-12 17:37:40,086 INFO org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService: disabled placement handler will be used, all scheduling requests will be rejected.
2022-12-12 17:37:40,087 INFO org.apache.hadoop.yarn.server.resourcemanager.AMSProcessingChain: Adding [org.apache.hadoop.yarn.server.resourcemanager.scheduler.constraint.processor.DisabledPlacementProcessor] tp top of AMS Processing chain.
2022-12-12 17:37:40,104 INFO org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: TimelineServicePublisher is not configured
2022-12-12 17:37:40,220 INFO org.eclipse.jetty.util.log: Logging initialized @2697ms to org.eclipse.jetty.util.log.Slf4jLog
2022-12-12 17:37:40,603 WARN org.apache.hadoop.security.authentication.server.AuthenticationFilter: Unable to initialize FileSignerSecretProvider, falling back to use random secrets. Reason: Could not read signature secret file: /home/hduser/hadoop-http-auth-signature-secret
2022-12-12 17:37:40,608 INFO org.apache.hadoop.http.HttpRequestLog: Http request log for http.requests.resourcemanager is not defined
2022-12-12 17:37:40,620 INFO org.apache.hadoop.http.HttpServer2: Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
2022-12-12 17:37:40,626 INFO org.apache.hadoop.http.HttpServer2: Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context cluster
2022-12-12 17:37:40,633 INFO org.apache.hadoop.http.HttpServer2: Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context logs
2022-12-12 17:37:40,633 INFO org.apache.hadoop.http.HttpServer2: Added filter RMAuthenticationFilter (class=org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter) to context static
2022-12-12 17:37:40,634 INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context cluster
2022-12-12 17:37:40,634 INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs
2022-12-12 17:37:40,634 INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static
2022-12-12 17:37:41,697 INFO org.apache.hadoop.yarn.webapp.WebApps: Registered webapp guice modules
2022-12-12 17:37:41,715 INFO org.apache.hadoop.http.HttpServer2: Jetty bound to port 8088
2022-12-12 17:37:41,719 INFO org.eclipse.jetty.server.Server: jetty-9.4.43.v20210629; built: 2021-06-30T11:07:22.254Z; git: 526006ecfa3af7f1a27ef3a288e2bef7ea9dd7e8; jvm 1.8.0_352-b08
2022-12-12 17:37:41,903 INFO org.eclipse.jetty.server.session: DefaultSessionIdManager workerName=node0
2022-12-12 17:37:41,903 INFO org.eclipse.jetty.server.session: No SessionScavenger set, using defaults
2022-12-12 17:37:41,915 INFO org.eclipse.jetty.server.session: node0 Scavenging every 660000ms
2022-12-12 17:37:41,984 WARN org.apache.hadoop.security.authentication.server.AuthenticationFilter: Unable to initialize FileSignerSecretProvider, falling back to use random secrets. Reason: Could not read signature secret file: /home/hduser/hadoop-http-auth-signature-secret
2022-12-12 17:37:41,996 INFO org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: Updating the current master key for generating delegation tokens
2022-12-12 17:37:42,004 INFO org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s)
2022-12-12 17:37:42,009 INFO org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: Updating the current master key for generating delegation tokens
2022-12-12 17:37:42,088 INFO org.eclipse.jetty.server.handler.ContextHandler: Started o.e.j.s.ServletContextHandler@53bd8fca{logs,/logs,file:///opt/hadoop/logs/,AVAILABLE}
2022-12-12 17:37:42,089 INFO org.eclipse.jetty.server.handler.ContextHandler: Started o.e.j.s.ServletContextHandler@63a270c9{static,/static,jar:file:/opt/hadoop/share/hadoop/yarn/hadoop-yarn-common-3.3.3.jar!/webapps/static,AVAILABLE}
2022-12-12 17:37:47,506 ERROR org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: RECEIVED SIGNAL 1: SIGHUP
2022-12-12 17:37:49,057 INFO org.eclipse.jetty.server.handler.ContextHandler: Started o.e.j.w.WebAppContext@64984b0f{cluster,/,file:///tmp/jetty-hadoop1-8088-hadoop-yarn-common-3_3_3_jar-_-any-6938409273500055326/webapp/,AVAILABLE}{jar:file:/opt/hadoop/share/hadoop/yarn/hadoop-yarn-common-3.3.3.jar!/webapps/cluster}
2022-12-12 17:37:49,086 INFO org.eclipse.jetty.server.AbstractConnector: Started ServerConnector@67c2e933{HTTP/1.1, (http/1.1)}{hadoop1:8088}
2022-12-12 17:37:49,086 INFO org.eclipse.jetty.server.Server: Started @11564ms
2022-12-12 17:37:49,087 INFO org.apache.hadoop.yarn.webapp.WebApps: Web app cluster started at 8088
2022-12-12 17:37:49,881 INFO org.apache.hadoop.ipc.CallQueueManager: Using callQueue: class java.util.concurrent.LinkedBlockingQueue, queueCapacity: 100, scheduler: class org.apache.hadoop.ipc.DefaultRpcScheduler, ipcBackoff: false.
2022-12-12 17:37:50,048 INFO org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 8033
2022-12-12 17:37:50,725 INFO org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl: Adding protocol org.apache.hadoop.yarn.server.api.ResourceManagerAdministrationProtocolPB to the server
2022-12-12 17:37:50,748 INFO org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Transitioning to active state
2022-12-12 17:37:50,730 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting
2022-12-12 17:37:50,758 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 8033: starting
2022-12-12 17:37:50,815 INFO org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore: Updating AMRMToken
2022-12-12 17:37:50,815 INFO org.apache.hadoop.yarn.server.resourcemanager.security.RMContainerTokenSecretManager: Rolling master-key for container-tokens
2022-12-12 17:37:50,815 INFO org.apache.hadoop.yarn.server.resourcemanager.security.NMTokenSecretManagerInRM: Rolling master-key for nm-tokens
2022-12-12 17:37:50,815 INFO org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: Updating the current master key for generating delegation tokens
2022-12-12 17:37:50,815 INFO org.apache.hadoop.yarn.server.resourcemanager.security.RMDelegationTokenSecretManager: storing master key with keyID 1
2022-12-12 17:37:50,815 INFO org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore: Storing RMDTMasterKey.
2022-12-12 17:37:50,842 INFO org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s)
2022-12-12 17:37:50,842 INFO org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: Updating the current master key for generating delegation tokens
2022-12-12 17:37:50,842 INFO org.apache.hadoop.yarn.server.resourcemanager.security.RMDelegationTokenSecretManager: storing master key with keyID 2
2022-12-12 17:37:50,842 INFO org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore: Storing RMDTMasterKey.
2022-12-12 17:37:50,919 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.nodelabels.event.NodeLabelsStoreEventType for class org.apache.hadoop.yarn.nodelabels.CommonNodeLabelsManager$ForwardingEventHandler
2022-12-12 17:37:51,809 INFO org.apache.hadoop.yarn.nodelabels.store.AbstractFSNodeStore: Created store directory :file:/tmp/hadoop-yarn-hduser/node-attribute
2022-12-12 17:37:52,198 INFO org.apache.hadoop.yarn.nodelabels.store.AbstractFSNodeStore: Finished write mirror at:file:/tmp/hadoop-yarn-hduser/node-attribute/nodeattribute.mirror
2022-12-12 17:37:52,198 INFO org.apache.hadoop.yarn.nodelabels.store.AbstractFSNodeStore: Finished create editlog file at:file:/tmp/hadoop-yarn-hduser/node-attribute/nodeattribute.editlog
2022-12-12 17:37:52,317 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.resourcemanager.nodelabels.NodeAttributesStoreEventType for class org.apache.hadoop.yarn.server.resourcemanager.nodelabels.NodeAttributesManagerImpl$ForwardingEventHandler
2022-12-12 17:37:52,320 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.placement.MultiNodeSortingManager: Starting NodeSortingService=MultiNodeSortingManager
2022-12-12 17:37:52,435 INFO org.apache.hadoop.ipc.CallQueueManager: Using callQueue: class java.util.concurrent.LinkedBlockingQueue, queueCapacity: 5000, scheduler: class org.apache.hadoop.ipc.DefaultRpcScheduler, ipcBackoff: false.
2022-12-12 17:37:52,445 INFO org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 8031
2022-12-12 17:37:52,475 INFO org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl: Adding protocol org.apache.hadoop.yarn.server.api.ResourceTrackerPB to the server
2022-12-12 17:37:52,476 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 8031: starting
2022-12-12 17:37:52,476 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting
2022-12-12 17:37:52,637 INFO org.apache.hadoop.util.JvmPauseMonitor: Starting JVM pause monitor
2022-12-12 17:37:52,709 INFO org.apache.hadoop.ipc.CallQueueManager: Using callQueue: class java.util.concurrent.LinkedBlockingQueue, queueCapacity: 5000, scheduler: class org.apache.hadoop.ipc.DefaultRpcScheduler, ipcBackoff: false.
2022-12-12 17:37:52,725 INFO org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 8030
2022-12-12 17:37:52,734 INFO org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl: Adding protocol org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB to the server
2022-12-12 17:37:52,735 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 8030: starting
2022-12-12 17:37:52,781 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting
2022-12-12 17:37:53,355 INFO org.apache.hadoop.ipc.CallQueueManager: Using callQueue: class java.util.concurrent.LinkedBlockingQueue, queueCapacity: 5000, scheduler: class org.apache.hadoop.ipc.DefaultRpcScheduler, ipcBackoff: false.
2022-12-12 17:37:53,367 INFO org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl: Adding protocol org.apache.hadoop.yarn.api.ApplicationClientProtocolPB to the server
2022-12-12 17:37:53,375 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 8032: starting
2022-12-12 17:37:53,376 INFO org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 8032
2022-12-12 17:37:53,379 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting
2022-12-12 17:37:54,354 INFO org.apache.hadoop.yarn.server.resourcemanager.ResourceTrackerService: NodeManager from node hadoop4(cmPort: 44399 httpPort: 8042) registered with capability: <memory:8192, vCores:8>, assigned nodeId hadoop4:44399
2022-12-12 17:37:54,354 INFO org.apache.hadoop.yarn.server.resourcemanager.ResourceTrackerService: NodeManager from node hadoop2(cmPort: 44199 httpPort: 8042) registered with capability: <memory:8192, vCores:8>, assigned nodeId hadoop2:44199
2022-12-12 17:37:54,354 INFO org.apache.hadoop.yarn.server.resourcemanager.ResourceTrackerService: NodeManager from node hadoop3(cmPort: 35453 httpPort: 8042) registered with capability: <memory:8192, vCores:8>, assigned nodeId hadoop3:35453
2022-12-12 17:37:54,361 INFO org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeImpl: hadoop4:44399 Node Transitioned from NEW to RUNNING
2022-12-12 17:37:54,366 INFO org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeImpl: hadoop3:35453 Node Transitioned from NEW to RUNNING
2022-12-12 17:37:54,367 INFO org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeImpl: hadoop2:44199 Node Transitioned from NEW to RUNNING
2022-12-12 17:37:54,433 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler: Added node hadoop4:44399 clusterResource: <memory:8192, vCores:8>
2022-12-12 17:37:54,446 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler: Added node hadoop3:35453 clusterResource: <memory:16384, vCores:16>
2022-12-12 17:37:54,460 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler: Added node hadoop2:44199 clusterResource: <memory:24576, vCores:24>
2022-12-12 17:37:54,917 INFO org.apache.hadoop.yarn.server.webproxy.ProxyCA: Created Certificate for OU=YARN-f8cdf15e-f7e8-4d9f-915a-b284670ec109
2022-12-12 17:37:55,034 INFO org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore: Storing CA Certificate and Private Key
2022-12-12 17:37:55,034 INFO org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Transitioned to active state
2022-12-12 17:37:55,042 ERROR org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted
2022-12-12 17:37:55,047 INFO org.eclipse.jetty.server.handler.ContextHandler: Stopped o.e.j.w.WebAppContext@64984b0f{cluster,/,null,STOPPED}{jar:file:/opt/hadoop/share/hadoop/yarn/hadoop-yarn-common-3.3.3.jar!/webapps/cluster}
2022-12-12 17:37:55,054 INFO org.eclipse.jetty.server.AbstractConnector: Stopped ServerConnector@67c2e933{HTTP/1.1, (http/1.1)}{hadoop1:8088}
2022-12-12 17:37:55,054 INFO org.eclipse.jetty.server.session: node0 Stopped scavenging
2022-12-12 17:37:55,055 INFO org.eclipse.jetty.server.handler.ContextHandler: Stopped o.e.j.s.ServletContextHandler@63a270c9{static,/static,jar:file:/opt/hadoop/share/hadoop/yarn/hadoop-yarn-common-3.3.3.jar!/webapps/static,STOPPED}
2022-12-12 17:37:55,055 INFO org.eclipse.jetty.server.handler.ContextHandler: Stopped o.e.j.s.ServletContextHandler@53bd8fca{logs,/logs,file:///opt/hadoop/logs/,STOPPED}
2022-12-12 17:37:55,058 INFO org.apache.hadoop.ipc.Server: Stopping server on 8032
2022-12-12 17:37:55,066 INFO org.apache.hadoop.ipc.Server: Stopping IPC Server listener on 8032
2022-12-12 17:37:55,066 INFO org.apache.hadoop.ipc.Server: Stopping IPC Server Responder
2022-12-12 17:37:55,070 INFO org.apache.hadoop.ipc.Server: Stopping server on 8033
2022-12-12 17:37:55,070 INFO org.apache.hadoop.ipc.Server: Stopping IPC Server listener on 8033
2022-12-12 17:37:55,071 INFO org.apache.hadoop.ipc.Server: Stopping IPC Server Responder
2022-12-12 17:37:55,072 INFO org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Transitioning to standby state
2022-12-12 17:37:55,072 WARN org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher: org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher$LauncherThread interrupted. Returning.
2022-12-12 17:37:55,073 INFO org.apache.hadoop.ipc.Server: Stopping server on 8030
2022-12-12 17:37:55,075 INFO org.apache.hadoop.ipc.Server: Stopping IPC Server listener on 8030
2022-12-12 17:37:55,076 INFO org.apache.hadoop.ipc.Server: Stopping IPC Server Responder
2022-12-12 17:37:55,078 INFO org.apache.hadoop.ipc.Server: Stopping server on 8031
2022-12-12 17:37:55,080 INFO org.apache.hadoop.ipc.Server: Stopping IPC Server listener on 8031
2022-12-12 17:37:55,083 INFO org.apache.hadoop.yarn.util.AbstractLivelinessMonitor: NMLivelinessMonitor thread interrupted
2022-12-12 17:37:55,083 ERROR org.apache.hadoop.yarn.event.EventDispatcher: Returning, interrupted : java.lang.InterruptedException
2022-12-12 17:37:55,085 ERROR org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Returning, interrupted : java.lang.InterruptedException: sleep interrupted
2022-12-12 17:37:55,085 INFO org.apache.hadoop.ipc.Server: Stopping IPC Server Responder
2022-12-12 17:37:55,086 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.activities.ActivitiesManager: org.apache.hadoop.yarn.server.resourcemanager.scheduler.activities.ActivitiesManager thread interrupted
2022-12-12 17:37:55,090 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: AsyncDispatcher is draining to stop, ignoring any new events.
2022-12-12 17:37:55,098 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: AsyncDispatcher is draining to stop, ignoring any new events.
2022-12-12 17:37:55,108 INFO org.apache.hadoop.yarn.util.AbstractLivelinessMonitor: AMLivelinessMonitor thread interrupted
2022-12-12 17:37:55,108 INFO org.apache.hadoop.yarn.util.AbstractLivelinessMonitor: org.apache.hadoop.yarn.server.resourcemanager.rmapp.monitor.RMAppLifetimeMonitor thread interrupted
2022-12-12 17:37:55,108 ERROR org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted
2022-12-12 17:37:55,108 INFO org.apache.hadoop.yarn.util.AbstractLivelinessMonitor: org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.ContainerAllocationExpirer thread interrupted
2022-12-12 17:37:55,109 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping ResourceManager metrics system...
2022-12-12 17:37:55,108 INFO org.apache.hadoop.yarn.util.AbstractLivelinessMonitor: AMLivelinessMonitor thread interrupted
2022-12-12 17:37:55,116 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: ResourceManager metrics system stopped.
2022-12-12 17:37:55,116 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: ResourceManager metrics system shutdown complete.
2022-12-12 17:37:55,116 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: AsyncDispatcher is draining to stop, ignoring any new events.
2022-12-12 17:37:55,117 INFO org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Transitioned to standby state
2022-12-12 17:37:55,117 INFO org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down ResourceManager at hadoop1/172.21.0.21
************************************************************/