Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Nest 8 + redis bug #82

Closed
herenickname opened this issue Jul 9, 2021 · 35 comments
Closed

Nest 8 + redis bug #82

herenickname opened this issue Jul 9, 2021 · 35 comments

Comments

@herenickname
Copy link

[Nest] 2697522 - 07/09/2021, 12:43:37 AM ERROR [ExceptionHandler] Nest can't resolve dependencies of the RedisCoreModule (Symbol(REDIS_MODULE_OPTIONS), ?). Please make sure that the argument ModuleRef at index [1] is available in the RedisCoreModule context.

Potential solutions:

  • If ModuleRef is a provider, is it part of the current RedisCoreModule?
  • If ModuleRef is exported from a separate @module, is that module imported within RedisCoreModule?
    @module({
    imports: [ /* the Module containing ModuleRef */ ]
    })
@aoxiang78
Copy link

Stopped here, there is no way to continue, do you have a temporary solution?

@rares-lupascu
Copy link

i personally duplicated this module into my local nestjs project

@aoxiang78
Copy link

@rares-lupascu Which files have you updated? Can you share it? I haven't solved this problem

@rares-lupascu
Copy link

@aoxiang78 ok might sound weird but just add this to redis-core.module.ts:

  1. import { ConfigModule } from '@nestjs/config';
  2. in the module definition .... change this
@Module({
  providers: [RedisService],
  exports: [RedisService],
})

to this (line 20):

@Module({
    providers: [RedisService],
    exports: [RedisService],
    imports: [ConfigModule],
})

@aoxiang78
Copy link

Solved it your way, thank you for your help @rares-lupascu

@brandongit2
Copy link

@rares-lupascu That doesn't seem to be working for me. Could you explain how adding ConfigModule to the imports solves NestJS looking for ModuleRef? That part confuses me.

@rares-lupascu
Copy link

@brandongit2 what I did was to create a local Redis module copying the contents of this repo into it (the lib folder) and applying the change I did at the top ... hope this helps

@alikil
Copy link

alikil commented Jul 13, 2021

I cannot use this solution at production build.
Still waiting for this library fix.

@PraveenKumarPalai
Copy link

@rares-lupascu Nice workaround for local 😎 !!

But for production, when can we expect a fix on this?

@bangbang93
Copy link

Keep watching. We are migrating to nestjs 8 and stuck on this.

@GyanendroKh
Copy link
Contributor

Instead of using ModuleRef to get the RedisClient, the RedisClient could be directly injected using @Inject(REDIS_CLIENT). And this solves the problem. I opened a pull request #85.

For now I'm using the library by directly downloading it from my Github Repo. Hope it will be released soon.

{
  "dependencies": {
    "nestjs-redis": "git+https://github.com/GyanendroKh/nestjs-redis.git#nest8-fix",
  }
}

@cdlaiqi
Copy link

cdlaiqi commented Jul 30, 2021

@rares-lupascu solved the problem, thank you

@nolawnchairs
Copy link

It's been over two years since the last release was published, so I personally consider this project abandoned.

@SampaioLeal
Copy link

It's been over two years since the last release was published, so I personally consider this project abandoned.

you mean the 1.3.3 4 months ago?

@Yurko-Fedoriv
Copy link

Adding here: actually, no code change is required, and lib just needs to be recompiled with updated dependencies.
If you simply copy lib/ into your src codebase and import from it instead of nestjs-redis app will work just fine.

That is why this "weird" solution works
And while this PR is valid, it is not needed too.

If you are using yarn, you probably can just install the package from github and import from lib/
npm will try to run prepublish and fail on non-existing yarn.
You can fiddle with postinstall command of your project to pull the dependency from gitlab.

I personally will just copy the lib/ into my codebase and hope the package will eventually get the update.

@MichaelTsengLZ
Copy link

MichaelTsengLZ commented Oct 13, 2021

This issue is still exist while installing with npm. Looks like you need to republish your npm package. Currently, I'm overcome this issue by changing the package.json

"nestjs-redis": "git+https://github.com/skunight/nestjs-redis.git",

image

@manlyman29
Copy link

@MichaelTsengLZ The issue is with CI/CD referring to github repo doesn't works and i couldn't find a workaround as of now. Really hope a contributor could raise the version and publish on npm, I am seriously stuck over this.

@lioralon326
Copy link

@MichaelTsengLZ The issue is with CI/CD referring to github repo doesn't works and i couldn't find a workaround as of now. Really hope a contributor could raise the version and publish on npm, I am seriously stuck over this.

Try upgrading to nest 8.0.9 and above, works there for me

@zackarychapple
Copy link

git+https://github.com/skunight/nestjs-redis.git

Confirmed that this worked for me as well. Hope to see this updated as using git+https is not as reliable for protecting against breaking changes.

@volodyad
Copy link

git+https://github.com/skunight/nestjs-redis.git

Confirmed that this worked for me as well. Hope to see this updated as using git+https is not as reliable for protecting against breaking changes.

You could connect to git commit "nestjs-redis": "git+https://github.com/skunight/nestjs-redis.git#939f33a7f9b252e872c6edb76a3a860ed053bacf", it will be safer at least

@pietrovismara
Copy link

@skunight any change of getting this fix in the latest on npm?

@Distortedlogic
Copy link

bump

@robertsLando
Copy link

robertsLando commented Jan 25, 2022

Today this seems to still be a problem...

Solution: npm install @liaoliaots/nestjs-redis

@Distortedlogic
Copy link

Distortedlogic commented Jan 26, 2022

@robertsLando src/index.ts(8,30): error TS2307: Cannot find module 'nestjs-redis' or its corresponding type declarations. your package did not work for me :/ but the git+https://github.com/skunight/nestjs-redis.git#939f33a7f9b252e872c6edb76a3a860ed053bacf one does work for me

@robertsLando
Copy link

@Distortedlogic you have to import @liaoliaots/nestjs-redis not nestjs-redis

@stefan030
Copy link

@Distortedlogic you have to import @liaoliaots/nestjs-redis not nestjs-redis

Still getting similar error after trying out liaoliaots' package.

Nest can't resolve dependencies of the Symbol(create-nestjs-middleware-module:options) (?, ConfigService). Please make sure that the argument RedisService at index [0] is available in the CoreModule context.

My nestjs version is 8.2.6

@stefan030
Copy link

As well as:
The redis client-provider 'Symbol(default)' was not found in the application context.

@Distortedlogic
Copy link

Distortedlogic commented Jan 31, 2022

@stefan030 read the docs... this redis module cannot just be dropped inplace of the other. The config options have a different structure.. also, the module is global only! so u cannot import RedisModule into other modules without reinstantiating it with forRoot or whichever func, instead since its global just put in AppModule, n inject the service. hope this helps!

@robertsLando
Copy link

robertsLando commented Jan 31, 2022

TLDR;

import { RedisModule } from '@liaoliaots/nestjs-redis';

/* ....  */
 RedisModule.forRoot({
      closeClient: true,
      config: {
        host: 'localhost',
        port: 6380,
      },
    }),

Docs: https://github.com/liaoliaots/nestjs-redis/blob/main/docs/v4/redis.md

auragit pushed a commit to auragit/nestjs-redis that referenced this issue Feb 8, 2022
@JustDoItSascha
Copy link

JustDoItSascha commented Feb 16, 2022

No release yet? Why is it closed? The error still exists...

@Halinen
Copy link

Halinen commented Feb 18, 2022

尝试用yarn更新了半天,最后直接把fix的版本clone了下来解决了- -

@JuKu
Copy link

JuKu commented Feb 18, 2022

@robertsLando Thank you very much! :)

epitone added a commit to clubpog/POG_server that referenced this issue Apr 4, 2022
1. 기존 cache-manager, cache-manager-ioredis 라이브러리 삭제
2. nestjs-redis 라이브러리에서 RedisCoreModule에서 참조 관련 문제가 발생함으로, dependencies 수정
3. 참고 skunight/nestjs-redis#82
epitone added a commit to clubpog/POG_server that referenced this issue Apr 4, 2022
## 작업사항
- 기존, NestJS에서 제공하는 cacheModule과 Redis를 함께 활용하는 방법을 구성했으나, 이렇게 활용하면 Redis의 많은 자료구조를 활용할 수 없는 문제가 있었음.
- Redis를 폭넓게 활용하기 위해서 Redis Client를 통해 Redis를 활용하는 방법을 구성함.
- nestjs-redis를 활용할 때, 순환 참조 문제가 발생함에 따라, 의존성 설정 변경 (skunight/nestjs-redis#82)
- 환경 변경을 통해, 즐겨찾기 추가, 삭제 API를 호출할 때, Redis 캐시가 적용되도록 변경
- cacheManager를 주입 받게 되어있던 테스트 환경도 수정


## 관계된 이슈, PR 
#104 #111 #113 

## 참고
https://www.npmjs.com/package/nestjs-redis
https://ably.com/blog/migrating-from-node-redis-to-ioredis
skunight/nestjs-redis#82
epitone added a commit to clubpog/POG_server that referenced this issue Apr 11, 2022
* [chore]: bull.js 사용을 위한 환경 설정 (#103)

* POG-143 [chore]: bull 의존성 설치
1. @nestjs/bull, bull, @types/bull 설치

* POG-143 [chore]: bull 의존성 설정

1. BullModule 설정

* POG-143 [chore]: Push 모노레포 설정 준비

* POG-143 [chore]: BullModule 설정

* POG-143 [chore]: BullQueue 환경 설정

1. BullQueue를 활용할 수 있도록 환경 설정

* POG-143 [chore]: nest-cli 모노레포 환경 추가

1. 서버를 실행할 때, 앱서버와 푸시서버를 나눠서 관리할 때, sourceRoot를 재설정할 필요가 있음

* [chore]: bull-monitor를 활용한 Queue 모니터링 구축 (#105)

1. @bull-monitor/express 의존성 설치를 통한 Queue 모니터링 구축 준비
2. BullMonitorModule, BullMonitorService 설정을 통한 Queue 모니터링 구축

* [chore]: CacheModule, in-memory 구성을 위한 의존성 설치  (#107)

- redis 적용을 위해 CacheModule 구성했음.
- CacheModule 적용을 위해 cache-manager, @types/cache-manager 설치
- CacheModule에 Redis 적용을 위해 cache-manager-ioredis, @types/cache-manager-ioredis 설치

* [feat]: 즐겨찾기 추가 API 호출 시, Redis 캐시 적용 (#110)

1. 만약 특정 소환사를 즐겨찾기 했을 때, 캐시에 데이터가 없으면, 전적 확인을 위한 Redis 캐시 추가
2. 임시적으로 캐시 만료 시간 제거, 캐싱 전략에 따라 만료 시간을 수정할 예정.
3. Redis 캐시에 저장하는 데이터는 삭제되어도 무관한 정보들임.
4. 만약 Redis 서버가 장애가 났을 때, 데이터가 삭제된다면, DB에 있는 데이터를 Redis에 저장하는 Cache Warming 작업을 진행해야 함. 
5. 만약, DB에 있는 소환사 전적 데이터가 최신 정보가 아닐 시에는, Riot API를 호출해서, 최신 데이터를 Redis에 저장해야 함.

#111

* [feat]: 즐겨찾기 삭제 API 호출 시, Redis 캐시 적용 (#112)

1. 아무도 즐겨찾기 하지 않는 소환사의 경우 Redis 캐시에서도 데이터가 삭제되게끔 설정
2. 테스트 코드 수정

* [fix]: 캐시를 Redis Client를 활용하는 방법으로 환경 변경 (#117)

## 작업사항
- 기존, NestJS에서 제공하는 cacheModule과 Redis를 함께 활용하는 방법을 구성했으나, 이렇게 활용하면 Redis의 많은 자료구조를 활용할 수 없는 문제가 있었음.
- Redis를 폭넓게 활용하기 위해서 Redis Client를 통해 Redis를 활용하는 방법을 구성함.
- nestjs-redis를 활용할 때, 순환 참조 문제가 발생함에 따라, 의존성 설정 변경 (skunight/nestjs-redis#82)
- 환경 변경을 통해, 즐겨찾기 추가, 삭제 API를 호출할 때, Redis 캐시가 적용되도록 변경
- cacheManager를 주입 받게 되어있던 테스트 환경도 수정


## 관계된 이슈, PR 
#104 #111 #113 

## 참고
https://www.npmjs.com/package/nestjs-redis
https://ably.com/blog/migrating-from-node-redis-to-ioredis
skunight/nestjs-redis#82

* [feat]: 즐겨찾기 추가, 취소 API 호출 시, Redis에 summonerId 관리 로직 구성 (#120)

1. 즐겨찾기 추가 API 호출 시, Redis Set 자료구조 방식으로, summonerId를 키로 하고, 값으로 입력받은 summonerId를 구성
2. 즐겨찾기 취소 API 호출 시, 아무도 즐겨찾기 하지 않는 사용자는 summonerId를 Set 키에서 제거하는 로직 구성
3. sadd, srem stub 로직 추가

* [feat]: Riot API와 비교해서, 전적이 변동됐을 때 로직 구성 (#121)

1. pushQueue에 EnQueue 작업 추가
2. 현재 Redis의 기록과 Riot API의 기록을 비교하는 로직 추가
3. 기록이 변경됐으면, Riot API에서 얻은 데이터를 Redis 기록에 수정하는 로직 추가
4. Queue 내용 추가
5. 불필요한 로직 제거

* [feat]: 일정 시간마다 푸시 보내기 위한 Task Schedule 설정 (#123)

1. Task Schedule을 위한 의존성 설치
2. 5분마다 푸시를 보낼 수 있도록 로직 구성

* [feat]: Topic을 활용한 FCM push 로직 구성 (#124)

1. FCM push 활용을 위한 firebase-admin 설정
2. json도 불러올 수 있게끔 tsconfig.json 변경
3. pushJob을 모듈 단위로 설정할 수 있게끔 설정
4. 푸시 테스트를 위해 필요 로직 주석 처리

* [chore]: TypeORM Quert Timeout 설정 (#126)

1. 슬로우 쿼리가 발생했을 때, 성능저하를 막기 위해 쿼리 시간이 특정 시간 이상 지속되는 경우 timeout으로 종료시키는 설정 추가
2. timeout 설정을 통해 장애 전파 최소화

* [fix]: 즐겨찾기 조회 API 쿼리 튜닝 및 인덱스 재설정 (#128)

## 작업사항
- 즐겨찾기 조회 API의 기존 쿼리는 1000만건 데이터를 조회할 때 응답속도가 약 1.5초가 걸리는 상황이었음.
- 데이터가 많아질수록 비효율적인 쿼리 방식이었음.
- FavoriteSummoner 테이블에 user_id 인덱스를 추가하여, Seq Scan이 아닌 Index Scan을 활용하도록 설정
- 쿼리 튜닝으로 인하여 1000만건 데이터를 조회할 때 1.5초에서 0.01초로 줄일 수 있었음. 
- 유니크 제약이 걸려있는 컬럼에 인덱스를 설정해서, 중복으로 인덱스가 설정되어있었던 상황이었고, 인덱스 중복을 제거함.

## 관계된 이슈, PR 
#75

* [fix]: favorite_summoner 테이블 인덱스 변경 (#129)


## 작업사항
1. 복합 인덱스 설정을 통한 쿼리 성능 향상

## 관계된 이슈, PR 
#75 

## 작업 내용
https://overcome-the-limits.tistory.com/666

* [feat]: Redis 장애 복구 API 추가 (#131)

## 작업사항
1. Redis 서버 장애 시, 데이터가 모두 사라질 경우를 대비해서, RDS를 활용하여 Redis에 데이터를 추가하는 작업 로직 추가

## 관계된 이슈, PR 
#115

* [chore]: 환경 변수 분리 (#132)

## 작업사항
1. 배포 전, development, production 별로 다르게 동작하도록 설정

## 관계된 이슈, PR 
#31
@ejhayes ejhayes mentioned this issue May 14, 2022
9 tasks
@ymc9
Copy link

ymc9 commented Jul 26, 2022

It turns out that for liaoliaots package RedisModule is by default global, you'll run into error if you import it into another module... Remove the import and the error is gone.

@celleb
Copy link

celleb commented Aug 18, 2022

@rares-lupascu Nice workaround for local 😎 !!

But for production, when can we expect a fix on this?

You can use https://www.npmjs.com/package/patch-package

npx patch-package nestj-redis

This will apply your changes every time you install or reinstall the package

@phillippelevidad
Copy link

phillippelevidad commented Apr 2, 2024

This issue is still exist while installing with npm. Looks like you need to republish your npm package. Currently, I'm overcome this issue by changing the package.json

"nestjs-redis": "git+https://github.com/skunight/nestjs-redis.git",

image

This still seems to be the best solution in 2024 (NestJS v10, apparently not supported by liaoliaots).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests