sickprodigy/home-server
1
0
Fork 0
Code Issues 15 Pull Requests Packages Projects Releases Wiki Activity

Youtube-DL-Material ; Crashing Heap out of Memory #24

New Issue
Closed
opened 2023-12-14 17:13:10 -05:00 by sickprodigy · 4 comments
sickprodigy commented 2023-12-14 17:13:10 -05:00
Owner
Copy Link

Error Code:

2023-12-14T22:12:20.025808086Z [47:0x545c7f0]    67049 ms: Scavenge 4025.4 (4119.4) -> 4024.9 (4129.9) MB, 9.6 / 0.0 ms  (average mu = 0.454, current mu = 0.398) allocation failure 

2023-12-14T22:12:20.025813250Z [47:0x545c7f0]    67091 ms: Scavenge 4031.5 (4129.9) -> 4030.3 (4130.6) MB, 10.5 / 0.0 ms  (average mu = 0.454, current mu = 0.398) allocation failure 

2023-12-14T22:12:20.025818536Z [47:0x545c7f0]    67749 ms: Scavenge 4032.6 (4130.6) -> 4031.5 (4151.1) MB, 645.9 / 0.0 ms  (average mu = 0.454, current mu = 0.398) allocation failure 

2023-12-14T22:12:20.025865898Z <--- JS stacktrace --->

2023-12-14T22:12:20.025881849Z FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory

2023-12-14T22:12:20.026164567Z  1: 0xb09980 node::Abort() [node /app/app.js]

2023-12-14T22:12:20.026531843Z  2: 0xa1c235 node::FatalError(char const*, char const*) [node /app/app.js]

2023-12-14T22:12:20.027036788Z  3: 0xcf784e v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [node /app/app.js]

2023-12-14T22:12:20.027534863Z  4: 0xcf7bc7 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [node /app/app.js]

2023-12-14T22:12:20.028085233Z  5: 0xeaf465  [node /app/app.js]

2023-12-14T22:12:20.028658351Z  6: 0xebf12d v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [node /app/app.js]

2023-12-14T22:12:20.029229304Z  7: 0xec1e2e v8::internal::Heap::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node /app/app.js]

2023-12-14T22:12:20.029777527Z  8: 0xe8336a v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationType, v8::internal::AllocationOrigin) [node /app/app.js]

2023-12-14T22:12:20.030398597Z  9: 0x11fc0b6 v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [node /app/app.js]

2023-12-14T22:12:20.032063271Z 10: 0x15f0b19  [node /app/app.js]

2023-12-14T22:12:33.842559131Z 2023-12-14T22:12:33: PM2 log: App [YoutubeDL-Material:0] exited with code [0] via signal [SIGABRT]

2023-12-14T22:12:33.845573862Z 2023-12-14T22:12:33: PM2 log: App [YoutubeDL-Material:0] starting in -fork mode-

2023-12-14T22:12:33.852859234Z 2023-12-14T22:12:33: PM2 log: App [YoutubeDL-Material:0] online

2023-12-14T22:12:34.903798182Z 2023-12-14T22:12:34.901Z INFO: Config items set using ENV variables.

2023-12-14T22:12:34.910555327Z (node:67) DeprecationWarning: uuidv4() is deprecated. Use v4() from the uuid module instead.

2023-12-14T22:12:34.910576030Z (Use `node --trace-deprecation ...` to show where the warning was created)

2023-12-14T22:12:35.347173185Z 2023-12-14T22:12:35.346Z INFO: YoutubeDL-Material v4.3.2 started on PORT 17442
Error Code: ``` 2023-12-14T22:12:20.025808086Z [47:0x545c7f0] 67049 ms: Scavenge 4025.4 (4119.4) -> 4024.9 (4129.9) MB, 9.6 / 0.0 ms (average mu = 0.454, current mu = 0.398) allocation failure 2023-12-14T22:12:20.025813250Z [47:0x545c7f0] 67091 ms: Scavenge 4031.5 (4129.9) -> 4030.3 (4130.6) MB, 10.5 / 0.0 ms (average mu = 0.454, current mu = 0.398) allocation failure 2023-12-14T22:12:20.025818536Z [47:0x545c7f0] 67749 ms: Scavenge 4032.6 (4130.6) -> 4031.5 (4151.1) MB, 645.9 / 0.0 ms (average mu = 0.454, current mu = 0.398) allocation failure 2023-12-14T22:12:20.025865898Z <--- JS stacktrace ---> 2023-12-14T22:12:20.025881849Z FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory 2023-12-14T22:12:20.026164567Z 1: 0xb09980 node::Abort() [node /app/app.js] 2023-12-14T22:12:20.026531843Z 2: 0xa1c235 node::FatalError(char const*, char const*) [node /app/app.js] 2023-12-14T22:12:20.027036788Z 3: 0xcf784e v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [node /app/app.js] 2023-12-14T22:12:20.027534863Z 4: 0xcf7bc7 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [node /app/app.js] 2023-12-14T22:12:20.028085233Z 5: 0xeaf465 [node /app/app.js] 2023-12-14T22:12:20.028658351Z 6: 0xebf12d v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [node /app/app.js] 2023-12-14T22:12:20.029229304Z 7: 0xec1e2e v8::internal::Heap::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node /app/app.js] 2023-12-14T22:12:20.029777527Z 8: 0xe8336a v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationType, v8::internal::AllocationOrigin) [node /app/app.js] 2023-12-14T22:12:20.030398597Z 9: 0x11fc0b6 v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [node /app/app.js] 2023-12-14T22:12:20.032063271Z 10: 0x15f0b19 [node /app/app.js] 2023-12-14T22:12:33.842559131Z 2023-12-14T22:12:33: PM2 log: App [YoutubeDL-Material:0] exited with code [0] via signal [SIGABRT] 2023-12-14T22:12:33.845573862Z 2023-12-14T22:12:33: PM2 log: App [YoutubeDL-Material:0] starting in -fork mode- 2023-12-14T22:12:33.852859234Z 2023-12-14T22:12:33: PM2 log: App [YoutubeDL-Material:0] online 2023-12-14T22:12:34.903798182Z 2023-12-14T22:12:34.901Z INFO: Config items set using ENV variables. 2023-12-14T22:12:34.910555327Z (node:67) DeprecationWarning: uuidv4() is deprecated. Use v4() from the uuid module instead. 2023-12-14T22:12:34.910576030Z (Use `node --trace-deprecation ...` to show where the warning was created) 2023-12-14T22:12:35.347173185Z 2023-12-14T22:12:35.346Z INFO: YoutubeDL-Material v4.3.2 started on PORT 17442 ```
sickprodigy commented 2023-12-14 17:38:08 -05:00
Author
Owner
Copy Link

Related issues:
https://github.com/Tzahi12345/YoutubeDL-Material/issues/676

Related issues: https://github.com/Tzahi12345/YoutubeDL-Material/issues/676
sickprodigy commented 2023-12-14 18:08:11 -05:00
Author
Owner
Copy Link

Related Environmental Variables possibly? Just need to widen node.js memory allocation.

https://github.com/Tzahi12345/YoutubeDL-Material/wiki/Environment-Variables

https://support.circleci.com/hc/en-us/articles/360009208393-How-Can-I-Increase-the-Max-Memory-for-Node-

The solution to run your Node.js app with increased memory is to start the process with an additional V8 flag: --max-old-space-size. You need to append your desired memory size in megabytes.

The following command will start the server.js with a memory limit of 8 GB.

node --max-old-space-size=8192 server.js

https://futurestud.io/tutorials/node-js-increase-the-memory-limit-for-your-process

These are supposedly the environment variables it's talking about above
https://github.com/Tzahi12345/YoutubeDL-Material/wiki/Configuration#items

Related Environmental Variables possibly? Just need to widen node.js memory allocation. https://github.com/Tzahi12345/YoutubeDL-Material/wiki/Environment-Variables https://support.circleci.com/hc/en-us/articles/360009208393-How-Can-I-Increase-the-Max-Memory-for-Node- The solution to run your Node.js app with increased memory is to start the process with an additional V8 flag: --max-old-space-size. You need to append your desired memory size in megabytes. The following command will start the server.js with a memory limit of 8 GB. node --max-old-space-size=8192 server.js https://futurestud.io/tutorials/node-js-increase-the-memory-limit-for-your-process These are supposedly the environment variables it's talking about above https://github.com/Tzahi12345/YoutubeDL-Material/wiki/Configuration#items
sickprodigy commented 2023-12-14 18:56:47 -05:00
Author
Owner
Copy Link

New Docker Compose config:

#version: "2.1"    # Outdated and not used in updated compose/portainer
services:
  ytdl_material:
    container_name: ytdl-material
#    hostname: youtube-dl-server      # if you need a quick second name possibly
    environment:
      UID: 1000
      GID: 1006
      ALLOW_CONFIG_MUTATIONS: 'true'
      NODE_OPTIONS: '--max-old-space-size=8192'
      ytdl_mongodb_connection_string: 'mongodb://sickprodigy:REDACTED@mongo:27017/'
      ytdl_use_local_db: 'false'
      write_ytdl_config: 'true'
      ytdl_title_top: 'Video Downloader'
    restart: always
#    depends_on: # Can only be used if in same compose file
#      - mongo-db
    volumes:
      - /srv/samba/media/Documents/prodigy/Coding-Programming/docker-containers/youtube-dl-material/appdata:/app/appdata
      - /srv/samba/media/Music/Youtube-DL:/app/audio
      - /srv/samba/media/Videos/Youtube-DL:/app/video
      - /srv/samba/media/Documents/prodigy/Coding-Programming/docker-containers/youtube-dl-material/subscriptions:/app/subscriptions
      - /srv/samba/media/Documents/prodigy/Coding-Programming/docker-containers/youtube-dl-material/users:/app/users
    ports:
      - "8998:17442"
    image: tzahi12345/youtubedl-material:latest
    stdin_open: true
    tty: true
    networks:
      homelab:
        aliases:
          - youtube-dl-server  # how to add multiple aliases
          - youtube-dl
          - ytdl
          - yt-dl-material
networks:    # I don't know why but you have to specify the network 2x. In service, and in this tag area..
  homelab:            # Sub-section, any network name or 'default' to edit default one created

    external: true      # This option causes compose to join the above network instead of making a _default one
New Docker Compose config: ``` #version: "2.1" # Outdated and not used in updated compose/portainer services: ytdl_material: container_name: ytdl-material # hostname: youtube-dl-server # if you need a quick second name possibly environment: UID: 1000 GID: 1006 ALLOW_CONFIG_MUTATIONS: 'true' NODE_OPTIONS: '--max-old-space-size=8192' ytdl_mongodb_connection_string: 'mongodb://sickprodigy:REDACTED@mongo:27017/' ytdl_use_local_db: 'false' write_ytdl_config: 'true' ytdl_title_top: 'Video Downloader' restart: always # depends_on: # Can only be used if in same compose file # - mongo-db volumes: - /srv/samba/media/Documents/prodigy/Coding-Programming/docker-containers/youtube-dl-material/appdata:/app/appdata - /srv/samba/media/Music/Youtube-DL:/app/audio - /srv/samba/media/Videos/Youtube-DL:/app/video - /srv/samba/media/Documents/prodigy/Coding-Programming/docker-containers/youtube-dl-material/subscriptions:/app/subscriptions - /srv/samba/media/Documents/prodigy/Coding-Programming/docker-containers/youtube-dl-material/users:/app/users ports: - "8998:17442" image: tzahi12345/youtubedl-material:latest stdin_open: true tty: true networks: homelab: aliases: - youtube-dl-server # how to add multiple aliases - youtube-dl - ytdl - yt-dl-material networks: # I don't know why but you have to specify the network 2x. In service, and in this tag area.. homelab: # Sub-section, any network name or 'default' to edit default one created external: true # This option causes compose to join the above network instead of making a _default one ```
sickprodigy commented 2025-06-04 18:17:15 -04:00
Author
Owner
Copy Link

If you subscribe to channels with over 1k episodes it will get out of heap. We have switched to tubearchivist and it has plex and jellyfin plugin to port the videos over. Has a shitty naming scheme but they are stuck on it.. whatever. using this mostly for quick downloading of videos.

If you subscribe to channels with over 1k episodes it will get out of heap. We have switched to tubearchivist and it has plex and jellyfin plugin to port the videos over. Has a shitty naming scheme but they are stuck on it.. whatever. using this mostly for quick downloading of videos.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Compat/Breaking
Breaking change that won't be backward compatible
Kind/Bug
Something is not working
Kind/Documentation
Documentation changes
Kind/Enhancement
Improve existing functionality
Kind/Feature
New functionality
Kind/Security
This is security issue
Kind/Testing
Issue or pull request related to testing
Priority
Critical
The priority is critical
Priority
High
The priority is high
Priority
Low
The priority is low
Priority
Medium
The priority is medium
Reviewed
Confirmed
Issue has been confirmed
Reviewed
Duplicate
This issue or pull request already exists
Reviewed
Invalid
Invalid issue
Reviewed
Won't Fix
This issue won't be fixed
Status
Abandoned
Somebody has started to work on this but abandoned work
Status
Blocked
Something is blocking this issue or pull request
Status
Need More Info
Feedback is required to reproduce issue or to continue work
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
sickprodigy
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sickprodigy/home-server#24
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?