Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

'compute' can throw for user using ldap #77

Closed
roblourens opened this issue Apr 5, 2024 · 3 comments · Fixed by #78
Closed

'compute' can throw for user using ldap #77

roblourens opened this issue Apr 5, 2024 · 3 comments · Fixed by #78

Comments

@roblourens
Copy link
Contributor

Original issue microsoft/vscode-remote-release#9649 (comment)

Here, https://github.com/cyjake/ssh-config/blob/162eaa36ad7a0ffce0a463322e50a1633f9bd478/src/ssh-config.ts#L172C15-L172C26

os.userInfo can throw for some users https://nodejs.org/api/os.html#osuserinfooptions

But it looks like User is optional here, or may not be needed, what do you think?

@cyjake
Copy link
Owner

cyjake commented Apr 7, 2024

can process.env.USER be used as a fallback? otherwise I'd have to add a asynchronous whoami call, which might impose more change

@cyjake cyjake closed this as completed in #78 Apr 8, 2024
@roblourens
Copy link
Contributor Author

roblourens commented Apr 8, 2024

Probably, but I'm not sure of the significance of having the username here. Thanks for the fix!

@cyjake
Copy link
Owner

cyjake commented Apr 9, 2024

whoa, the commit references looks overwhelming, sorry about that, was fighting with code coverage (yet still failed). The username is needed to calculate Match directives (might be referenced in the criteria).

i'll release a patch version tonight.

justinmk3 added a commit to justinmk3/aws-toolkit-vscode that referenced this issue Jul 9, 2024
Problem:
Both AWS Toolkit and Amazon Q fail to start when the effective user
account is defined externally (e.g. by AD/LDAP). aws#5277

    [error] SystemError: A system error occurred: uv_os_get_passwd returned ENOENT (no such file or directory)

Bisected to aws#5215 which calls os.userInfo in getUsername (env.ts).

When the effective user ID does not have a matching entry in
/etc/passwd, userInfo throws an exception. Discussed in:
- microsoft/vscode-remote-release#9649
- cyjake/ssh-config#77

NOTE: Recent restructuring in aws@c21dce1 appears to retain this bug.

1. Login as a user that is synced to AD/LDAP.
2. Install AWS Toolkit and Amazon Q
3. Errors in `~/.config/Code/logs/<date and time>/window1/exthost/exthost.log`
   ```
   2024-07-08 12:09:54.179 [error] Activating extension amazonwebservices.amazon-q-vscode failed due to an error:
   2024-07-08 12:09:54.179 [error] SystemError: A system error occurred: uv_os_get_passwd returned ENOENT (no such file or directory)
           at new SystemError (node:internal/errors:257:5)
           at new NodeError (node:internal/errors:368:7)
           at Object.userInfo (node:os:365:11)
   ```

Solution:
- Move getUserInfo/getUsername into fs.ts since they depend on the
  filesystem.
- If userInfo fails, fallback to (in order of precedence):
  - process.env.USER
  - getUserHomeDir() directory name
  - "unknown-user"
justinmk3 added a commit to justinmk3/aws-toolkit-vscode that referenced this issue Jul 9, 2024
Problem:
Both AWS Toolkit and Amazon Q fail to start when the effective user
account is defined externally (e.g. by AD/LDAP). aws#5277

    [error] SystemError: A system error occurred: uv_os_get_passwd returned ENOENT (no such file or directory)

Bisected to aws#5215 which calls os.userInfo in getUsername (env.ts).

When the effective user ID does not have a matching entry in
/etc/passwd, userInfo throws an exception. Discussed in:
- microsoft/vscode-remote-release#9649
- cyjake/ssh-config#77

Steps to reproduce:
1. Login as a user that is synced to AD/LDAP.
2. Install AWS Toolkit and Amazon Q
3. Errors in `~/.config/Code/logs/<date and time>/window1/exthost/exthost.log`
   ```
   2024-07-08 12:09:54.179 [error] Activating extension amazonwebservices.amazon-q-vscode failed due to an error:
   2024-07-08 12:09:54.179 [error] SystemError: A system error occurred: uv_os_get_passwd returned ENOENT (no such file or directory)
           at new SystemError (node:internal/errors:257:5)
           at new NodeError (node:internal/errors:368:7)
           at Object.userInfo (node:os:365:11)
   ```

Solution:
- Move getUserInfo/getUsername into fs.ts since they depend on the
  filesystem.
- If userInfo fails, fallback to (in order of precedence):
  - process.env.USER
  - getUserHomeDir() directory name
  - "unknown-user"
justinmk3 added a commit to aws/aws-toolkit-vscode that referenced this issue Jul 9, 2024
Problem:
Both AWS Toolkit and Amazon Q fail to start when the effective user
account is defined externally (e.g. by AD/LDAP). #5277

    [error] SystemError: A system error occurred: uv_os_get_passwd returned ENOENT (no such file or directory)

Bisected to #5215 which calls os.userInfo in getUsername (env.ts).

When the effective user ID does not have a matching entry in
/etc/passwd, userInfo throws an exception. Discussed in:
- microsoft/vscode-remote-release#9649
- cyjake/ssh-config#77

Steps to reproduce:
1. Login as a user that is synced to AD/LDAP.
2. Install AWS Toolkit and Amazon Q
3. Errors in `~/.config/Code/logs/<date and time>/window1/exthost/exthost.log`
   ```
   2024-07-08 12:09:54.179 [error] Activating extension amazonwebservices.amazon-q-vscode failed due to an error:
   2024-07-08 12:09:54.179 [error] SystemError: A system error occurred: uv_os_get_passwd returned ENOENT (no such file or directory)
           at new SystemError (node:internal/errors:257:5)
           at new NodeError (node:internal/errors:368:7)
           at Object.userInfo (node:os:365:11)
   ```

Solution:
- Move getUserInfo/getUsername into fs.ts since they depend on the
  filesystem.
- If userInfo fails, fallback to (in order of precedence):
  - process.env.USER
  - getUserHomeDir() directory name
  - "unknown-user"
mvcninja added a commit to Bit-Quill/aws-toolkit-vscode that referenced this issue Jul 12, 2024
* docs: marketplace page #5113

* fix(build): husky "install command is deprecated" #5114

Problem:
husky prints warning during "npm install":

    > [email protected] prepare
    > ts-node ./scripts/prepare.ts
    install command is deprecated

Solution:
Migrate as described in https://github.com/typicode/husky/releases/tag/v9.0.1

* fix(amazonq): rename "Send to Amazon Q" #5112

* changelog

* feat(amazonq): display auth notification #5097

- Notification are not displayed if user is logged in and has an active connection
- Notifications are only displayed once.

display Auth Notification...
- When a user closes the Amazon Q chat panel and
- User has not performed any auth action

display Error Notification...
- When a user closes the Amazon Q chat panel and
- A user performs auth Action though is not logged it.
Assumption: User in this scenario might not necessarily be in error state.

* cleanup: remove empty file (#5117)

* Release 1.8.0

* Release 3.9.0

* Update version to snapshot version: 3.10.0-SNAPSHOT

* Update version to snapshot version: 1.9.0-SNAPSHOT

* web: setup web mode for Q inline suggestions (#5046)

* docs: Shared vs Common file keyword

Read the doc, this will impact how we name and organize modules.

Signed-off-by: Nikolas Komonen <[email protected]>

* refactor: extensionShared -> extensionCommon naming

use the "common" keyword instead of "shared.

Signed-off-by: Nikolas Komonen <[email protected]>

* refactor: rename amazonq extensionShared -> extensionCommon

Signed-off-by: Nikolas Komonen <[email protected]>

* refactor: get Amazon Q web running without errors on load

This fixes a few things:
- Non web compatible code being bundled in due to imports. This was fixed by moving the imported code in to a web compatible module which stopped incompatible transitive dependencies from being part of the final bundle
- renamed and organized files in to a 'common' or 'shared' file to indicate what is and isn't compatible in the web
- do not show the amazon q walkthrough if we are in web, since Q is not availabe in web mode yet anyway
- Q status bar, then sign in will immediately sign you in to BiD. This is temporary for development purposes.

Signed-off-by: Nikolas Komonen <[email protected]>

* refactor: show amazon q walkthrough

Moves the show amazon q walkthrough to a single place instead of spreading it around

Signed-off-by: Nikolas Komonen <[email protected]>

* minor fixes

Signed-off-by: Nikolas Komonen <[email protected]>

* refactor: make timeout work in web

Problem:

`globals.clock.setTimeout()` creates a timeout but returns a different
result depending on Node.JS vs Web. In Node.JS it returns an object, but
in Web it returns a number.

Some places call `refresh()` on the result, this works in Node.JS but fails
in browser.

Solution:

Update the Timeout class so that when `refresh()` is called it figures out
which environment we are in and does the appropriate refresh.

Also, update functionUtils to use the `Timeout` class instead of directly using
globals.clock.setTimeout()

Signed-off-by: Nikolas Komonen <[email protected]>

* web: disable Q view in web mode

Signed-off-by: Nikolas Komonen <[email protected]>

* fix import

Signed-off-by: Nikolas Komonen <[email protected]>

* formatting

Signed-off-by: Nikolas Komonen <[email protected]>

---------

Signed-off-by: Nikolas Komonen <[email protected]>

* fix(build): package task fails if commit id starts with zero #5118

Problem:
package task fails if commit id starts with zero.

    ERROR  Invalid extension version '3.9.0-0238109'

semver [disallows](https://semver.org/#spec-item-9) leading zero for
prerelease. This is why `git describe` always adds a "g" prefix, for
example.

Solution:
Prefix the prerelease string with "g", similar to `git describe`.

* fix: only install Q once for previous users (#5125)

Problem: Q installs whenever a past connection is detected. If users don't log out of that connection, then they will forever have Q auto-installed on startup even if they don't want it.
Solution: Only install once, then store to global state that we did.

- Cleanup the install code a bit
- Use the same global state key for dismissing the install q notification.

Fixes https://github.com/aws/aws-toolkit-vscode/issues/4898

* docs: marketplace #5119

* fix: unreliable test "transformByQ WHEN update job history..." #5135

Problem:
unreliable test:

      1 failing
      1) transformByQ
           WHEN update job history called THEN returns details of last run job:

          AssertionError [ERR_ASSERTION]: Expected values to be strictly deep-equal:
    + actual - expected ... Lines skipped

      {
        'abc-123': {
    +     duration: '2 sec',
    -     duration: '0 sec',
          projectName: 'test-project',
    ...
          status: 'COMPLETED'
        }
      }
          + expected - actual

           {
             "abc-123": {
          -    "duration": "2 sec"
          +    "duration": "0 sec"
               "projectName": "test-project"
               "startTime": "05/03/24, 11:35 AM"
               "status": "COMPLETED"
             }

          at Context.<anonymous> (d:\a\aws-toolkit-vscode\aws-toolkit-vscode\packages\core\src\test\codewhisperer\commands\transformByQ.test.ts:182:16)

Solution:
Use a narrower assertion. The important property is still tested, the
other properties are not relevant for this test.

fix https://github.com/aws/aws-toolkit-vscode/issues/5134

* fix(login): pre-select auth option for the user #5090

Problem
Users have to manually select one of the auth options for Login

Solution
To reduce the friction and one click, we pre-select the first option by default.

Note: In case of existing logins, we still select the default option as we are in process of separating out auth.

* fix(amazonq): remember "Pause Auto-Suggestions" after IDE restart #5126

Problem
IDE restart will re-enable auto trigger, even if user selected "Pause Auto-Suggestions".

Solution
Do not re-enable autosuggestions on startup.

* fix(amazonq): update copy for builder id #5137

* fix(amazonq): handle proxy-related error cases #5132

Problem
We want to give users more helpful error message for proxy-related errors they may experience.

Solution
Show users pre-approved text when these proxy-related errors happen.

* feat(amazonq): reauth enforces amazon q scopes only

**Session split commit**

Problem: Amazon Q will continue to use connections it already has. These connections may contain scopes from toolkit.

Solution: Reauth with only amazon q scopes once the connection expires.

* fix(dev mode): startup fails on invalid dev.logfile #5146

Problem:
Extension fails to start if the "aws.dev.logfile" setting returns
nonsense. `settings.getSection().get()` is a very thin wrapper around
`vscode.workspace.getConfiguration()`, so it doesn't have the type
checking and coercion of the `Settings` class.

Solution:
Check the type.

* fix(telemetry): guard for string overflow #5131

Problem
Emitting telemetry items that are too long.

Solution
Delete values if they are too long.

* fix(amazonq security scan): disable Auto-Scans for plaintext #5133

Problem
Exceptions are thrown for auto-scans with the plaintext language from server side.

Solution
Disabling auto-scans for plaintext as the issue needs service fix.

* chore: upgrade codewhisperer-streaming client #5147

Problem
Recent API changes.

Solution
Update the streaming client.

* refactor(dev): update dev menu for separate auth sessions (#5142)

**Split session commit**

Dev menu will now access individual extensions auth.

* fix(amazonq): update walkthrough

- Make inline suggestion as the first item for walkthrough
- Do not redirect user to Try inline suggestion example
- Add CTA and directly open an example
- add example for security scan in walkthrough

* fix(security scan): scan during user input causes slow performance #5129

Problem
- SecurityScan triggers `getDirSize()` function to calculate
  `codeScanProjectBytes` for telemetry for both `Auto-scans` and `Project
  scans`.

Solution
- Disable the `codeScanProjectBytes` calculation for `Auto-scans`: Since the
  `codeScanProjectBytes` parameter is optional for `Auto-scans` and is affecting
  the performance, it makes sense to disable the calculation for this scenario.
  This will help improve the overall performance of the Auto-scans.
- Enable the `codeScanProjectBytes` calculation for `Project scans`: For
  `Project scans`, you should continue to calculate the `codeScanProjectBytes`
  using the `getDirSize()` function, as this information is likely valuable for
  the telemetry.

* fix(codecatalyst): always send heartbeat activity

Problem
Toolkit requires a connection / auth session token to send heartbeat activity:
https://github.com/aws/aws-toolkit-vscode/blob/4512e2052ec29bed0b11184e606b3937daa99346/packages/core/src/codecatalyst/devEnv.ts#L73-L94
This means if the user is not connected, the dev env will time out.

Solution:
- Always send heartbeat activity (except when we know that `inactivityTimeoutMinutes=0`).
- If `inactivityTimeoutMinutes` is unknown, assume it is 5 minutes.

* feat(codecatalyst): "will stop soon" message disappears

Problem:
The "dev env will stop soon" message disappears after a few seconds.

Solution:
Set the message to be modal. Because auto-shutdown is imminent, this
message is important and should request attention.

* feat(settings): aws.dev.devenvTimeoutMs

Allow overriding devenv timeout in dev-mode.

* refactor: reduce indirection, verbosity

- improve logging

* fix(codecatalyst): setTimeout() called with negative interval

* fix(codecatalyst): short timeout may result in redundant warnings

* refactor(codecatalyst): simplify

Merge Message into InactivityMessage, their separation complicates the
logic, and adds indirection, for no benefit.

* feat(codecatalyst): improve logging

* feat(auth): forget about Amazon Q connections in toolkit (#5144)

**Separate sessions commit**

If the Toolkit has a connection with Amazon Q scopes, it must be in use by Amazon Q. We will forget these in Toolkit's auth. They will remain on disk. This will log out Q + Toolkit auth connections in Toolkit only.

* refactor(codecatalyst): cleanup auth code, add scope check (#5145)

- Remove some unused auth code
- Add CodeCatalyst reauth function that verifies we are only using codecatalyst scopes.

* techdebt: remove almost all of the tryRegister commands      (#5128)

* techdebt: remove almost all of the tryRegister commands

Problem:
- tryRegister is a hack to allow settings that get activated in the amazonq extension to run in core

Solution:
- setup the amazonq project to run unit tests
- Add "test" script to amazonq package.json
- Move amazonq, feature dev, gumby, codewhisperer, codewhispererChat unit tests from core to amazonq. This requires us to export any required dependencies from their respective source folders
- remove tryRegister in as many places as possible. See "Other notes" for the reason why we couldn't get rid of them all

Other notes:
- A few test files need to override vscode api in order to function correctly. Those test files have been left in core because the overriding wasn't working from a seperate package. tryRegister remains for a few of those files that are left behind and still need it
- A fallback for timer was needed when moving tests from core -> amazonq. It's unclear why these changes forced that fallback

* feat(amazon q transform): user can view build log #5140

Enable customers to view the server-side build logs that are generated for
a code transform after a project has been submitted. The build log will be
downloaded locally, and the option is given to view it in the chat.

* fix(featureDev): spinner displays even on failure #5148

Problem:
Steps to reproduce:
- Get the Latest Q plugin for VsCode
- Open the VsCode without opening any project or workspace
- Open Q, and type /dev
- In Dev chat tab, type sometinng and hit enter/Send button
- You'll see that `Generating your anwer...` response is still there even though there is an error message below that.

Solution:
If there is an error, don't show the "spinner" / loading message.

* fix(logging): "View Logs" fails #5164

* fix(codecatalyst): avoid self-triggered UserActivity #5152

Problem:
UserActivity may fire when the extension writes to its own logfile.

Solution:
Special-case the `onDidOpenTextDocument` event.

* doc: vscode.dev testing local build (#5159)

Doc to explain how to test your local development build in
vscode.dev

Signed-off-by: Nikolas Komonen <[email protected]>

* fix(login): show overflow text as ellipsis #5156

* auth: add auth for web mode (#5165)

Problem:
- We currently can't auth on web mode. Device code doesn't work because of cors and we can't start a local server without remote compute

Solution:
- Temporarily redirect to a (hopefully) unused port and send the code there. This unblocks testing web mode

* fix(auth): improve logging

* refactor(telemetry): use getClientId in setupTelemetryId

Problem:
setupTelemetryId duplicates the logic of getClientId

Solution:
delegate to getClientId

* fix: include Toolkit UserAgent in Q, OIDC/SSO requests

Example headers:

    user-agent: aws-sdk-js/3.574.0 ua/2.0 os/darwin#23.5.0 lang/js md/nodejs#20.9.0 api/sso-oidc#3.574.0 AmazonQ-For-VSCode/1.7.0/1.90.1-ClientId/96be923b-…
    x-amz-user-agent: aws-sdk-js/3.574.0 AmazonQ-For-VSCode/1.7.0/1.90.1-ClientId/96be923b-…

* telemetry: emit aws_refreshCredentials on success

Problem:
`aws_refreshCredentials` is only emitted on failure (except in the
deprecated `loginManager.ts`). This leads to skewed results and also
lacks visibility into `sessionDuration` for non-failure cases.

Solution:
Also emit `aws_refreshCredentials` on success.

* telemetry: set reasonDesc field on failure

Problem:
InvalidGrant errors are too generic. The message contains disambiguating
details.

Solution:
Set (partial, truncated) error message in the `reasonDesc` field.
Prefer the non-standard `error_description` field (if found), which is
provided by the OIDC service.

* telemetry: set requestId for all Failed metrics

Problem:
requestId is missing on login/auth-related metrics.

Solution:
- Always set requestId on failed metrics.
- Also set requestId on success metrics for:
  - aws_loginWithBrowser
  - aws_refreshCredentials

* [split sessions] feat(auth): update api for split sessions (#5150)

* feat(auth): update api for split sessions

**Separate sessions commit**

- Add new API to Toolkit to "declare" and "undeclare" a connection. Unlike the other API methods, this will inform the toolkit of existing connections without accepting them into its auth environment. This allows it to use params from other connections, e.g. start url region. It does not persist.
    - Retain original API because it may be beneficial in the future for other extensions/use cases
- Change Amazon Q's usage of the API to declare instead of direct session sharing.
- onDidDeleteConnection auth hook now sends out the connection profile it used to store.

* type and hash updates

* anything commit to fix github ci?

* fix(auth): forgetConnection emits old value for onDidDeleteConnection

* perf(amazon q security scan): frequent scans cause delays #5160

Problem
We are seeing Throttling for file scans and scan accuracy for files.

Solution
Increasing debounce period to 15 seconds

* telemetry: set requestId field on failure (#5168)

* telemetry: missing reasonDesc for aws_refreshCredentials (#5170)

* fix: remove redirectUri since it's no longer needed (#5171)

Problem:
- redirectUri used to be used to redirect back to vscode but we are no longer using that

Solution:
- remove it

* Release 3.10.0

* Release 1.9.0

* Update version to snapshot version: 3.11.0-SNAPSHOT

* Update version to snapshot version: 1.10.0-SNAPSHOT

* feat(errors): surface error_description

Problem:
Some services such as OIDC supply the non-standard `error_description`
SDK field on some errors. If it is present, it contains more useful
information than the default `message` field. But our error handling
does not surface this field.

ref https://github.com/aws/aws-toolkit-jetbrains/commit/cc9ed87fa9391dd39ac05cbf99b4437112fa3d10

Solution:
Introduce `getErrorMsg()` and use it in `resolveErrorMessageToDisplay()`.

* refactor(auth): remove sharing of direct sessions across extensions (#5169)

* refactor(auth): remove sharing of direct sessions across extensions

**Separate sessions commit**

Instead of sharing sessions directly (e.g. with tokens, logout logs out of both extensions, etc), we will just share SSO parameters across extensions. The auth can re-use starturl and region but will build its own connection.
Existing connections UI remains for this purpose.

- Remove unused auth and API code. This includes special handling logic that imports direct sessions from the other extension.
- Builder ID will not be displayed as an existing connection because the parameters for this are already known. Users can just log in with buidler ID normally.
- Misc fixes in comments, log statements.

* refactor: rename "existing" to "imported" logins

* feat(auth): re-add checks for existing start urls in same extension

* fix(errors): findPrioritizedAwsError stops at first AWSError

Problem:
`findPrioritizedAwsError` does not search the full cause-chain for the
best possible error.

Solution:
- Search the full cause-chain.
- Rename to `findBestErrorInChain`.

* fix(auth): login page displays connections that already exist for that service (#5186)

Problem: We do not allow the SSO form to continue if the user tries putting in an IdC start url that already exists in the same extension. However, this check isn't in place for displaying connections from the other extension. This means users can duplicate the connection start url in the extension and caused undefined behavior.

Solution: Filter out existing connections from other extensions on what connections are in the current extension.

* feat(ui): show login prompt for split sessions (#5182)

**Separate sessions commit**

- If user is logged out of toolkit due to extension session splitting, display a prompt that opens the sign in page when clicked.
- Dismissing or clicking the button dismisses it permanently.
- The prompt will continue to show across restarts until it is dismissed, the button is pressed, or the user signs into toolkit.
- The prompt will display only once, regardless of how many connections are logged out (forgotten).
- The sign in button will go to the login page with the proper scopes, i.e.g if codecatalyst was signed out, go to code catalyst login. Otherwise go to explorer only login.

* fix(featureDev): file rejections for files outside of src/ (#5190)

* fix(featureDev): Fix file rejections for files outside of src/

Problem:
- When the given path for a file starts with ./ it doesn't show the file actions and not triggering the file click. Now it uses the original file path check

Solution:
- Update mynah ui

* fix(featureDev): update welcome message #5191

* Always use device authorization in e2e t (#5106)

### Problem
- Auth now uses authorization grant flow by default but that ends up causing 2 issues in e2e tests:
    - We need to automatically press the "trust external url" popup created when we attempt to open the authorization url otherwise we get "Error: DialogService: refused to show dialog in tests. Contents: Do you want Code to open the external website?" when running the e2e tests
    - We don't support authorization grant flow in the auth lambda

### Solution
- Just use the device code auth flow until we support the auth grant flow

* feat(login): display tooltips during keyboard navigation #5184

* fix(amazonq security scan): avoid project scan timeouts #5183

Problem:
- VSC triggers `getDirSize()` function to calculate `codeScanProjectBytes` for
  telemetry for `Project scans`.

Solution:
- Removing the `codeScanProjectBytes` calculation for `Project-scans`: Since the
  `codeScanProjectBytes` parameter is optional and is affecting the performance
  of CreatePayload. This will help improve the overall performance of the
  Auto-scans.

* fix(amazonq security scan): increase max payload #5187

Problem:
Security Scans timing out due to payload size limit.

Solution:
Increase in payload size limit from `500MB` to `1GB`.

* telemetry: featureDev #5179

Problem
For testing ABTesting configuration and infrastructure, is needed to start sending telemetry events so RTS. This will create the basic implementation to track ABTests for featureDev.

Solution
Add the new event model to the service schema, Add a sendTelemetryEvent request when a conversation is created.

* fix(codecatalyst): dev env logged out on load (#5196)

* fix(codecatalyst): dev env logged out on load

Problem: We forget all connections without all codecatalyst scopes only (aws account + coca scopes). However, the stored credentials in dev environments do not have the account scopes, so we are forgetting it.

Solution: Check for Q scopes explicitly.

* Update packages/core/src/extension.ts

Co-authored-by: Justin M. Keyes <[email protected]>

---------

Co-authored-by: Justin M. Keyes <[email protected]>

* fix: remove userApproachTelemetry feature dev log when telemetry is disabled (#5197)

Problem:
- This debug log is misleading when telemetry is disabled

Solution:
- Only use the debug lob when telemetry is enabled

* fix(codecatalyst): split session auth issues (#5199)

Fixes:
- Adds 'source' to telemetry for logout prompt
- If in dev env, fall back to MDE credentials if logged out (due to sharing connection with Q)
- Bug where signing in to Amazon Q in dev env will add coca scopes (and otherwise share the underlying MDE credentials)
  - Amazon Q was also importing the MDE creds under the hood, so it was being detected as an existant connection. This means any login method will use that instead of creating a new connection.

* auth: ignore vscode proxy errors on token refresh #5198

We were seeing an error in `aws_refreshCredentials` metric.
This error was networking related when the user has setup a proxy
in vscode, but they were not connected to it. Maybe they didn't
connect to VPN.

We typically catch network errors during the SSO token refresh process
so that the session does not become invalidated on it. We do not catch
all errors, but will add them in incrementally as we see them in telemetry.
This proxy error was not caught and caused sessions to become invalidated prematurely.

Solution

Treat this VSCode Proxy Error as a network error so that we ignore
it during token refresh

Signed-off-by: Nikolas Komonen <[email protected]>

* fix(amazonq): code blocks with typewriter text inside list items #5201

Problem
- `<span class='typewriter-part>...</span>` shows up in contents within list items.
- Chat card component reordering and update structure is broken sometimes depending on update type.

Solution
- Streaming content component wrapped out from the chat-item-card and builded as a separate component
- Typewriter animation injection removed from the markdown parsing process
- Typewriter animation injection added to node process state with a check if the node type is TEXT, then we're adding the animation wrapper.

[MynahUI 4.13.0](https://github.com/aws/mynah-ui/releases/tag/v4.13.0)

* fix(amazonq): update login screen and auth notification (#5188)

* fix(amazonq): update login screen and auth notification

- Update Styling for Login Page and Reauth page
- Show notification once if a user is not logged in when Amazon Q extension activates.

* Update packages/amazonq/.changes/next-release/Bug Fix-9130bd3b-edc4-4749-8c96-bd9a4fb9f03c.json

---------

Co-authored-by: Maxim Hayes <[email protected]>

* fix(amazonq): errors are not surfaced #5181

Problem:
- amazonq main does not use activateCommon, so it is missing some common logic.
    - does not define a handler for bubbled-up errors, so errors are not displayed in a user-facing message.
- `findBestErrorInChain()` discards non-AWSError, so filesystem errors are not surfaced.

Solution:
- call `registerCommandErrorHandler` in amazonq init.
    - TODO: can amazonq init share common init w/ toolkit?
- improve `findBestErrorInChain()` so that it finds filesystem errors.

* fix(amazonq): "Failed to run: refreshConnectionCallback" #5205

Problem:
command sometimes shows an error during teardown:

    Failed to run command: aws.amazonq.refreshConnectionCallback: Cannot
    read properties of undefined (reading 'scopes')

Solution:
There may be a race somewhere, when we update the connections list. To
mitigate, handle undefined AwsConnection.

* telemetry(amazonq security scan): set reasonDesc #5194

Problem
Telemetry error messages are not consistent in VSC and JB

Solution
Improving telemetry error messages to keep consistent in both IDE's

* Release 3.11.0

* Release 1.10.0

* Update version to snapshot version: 3.12.0-SNAPSHOT

* Update version to snapshot version: 1.11.0-SNAPSHOT

* build(deps-dev): bump the npm_and_yarn group across 1 directory with 3 updates (#5185)

Bumps the npm_and_yarn group with 3 updates in the / directory: [braces](https://github.com/micromatch/braces), [express](https://github.com/expressjs/express) and [ws](https://github.com/websockets/ws).


Updates `braces` from 3.0.2 to 3.0.3
- [Changelog](https://github.com/micromatch/braces/blob/master/CHANGELOG.md)
- [Commits](https://github.com/micromatch/braces/compare/3.0.2...3.0.3)

Updates `express` from 4.18.1 to 4.19.2
- [Release notes](https://github.com/expressjs/express/releases)
- [Changelog](https://github.com/expressjs/express/blob/master/History.md)
- [Commits](https://github.com/expressjs/express/compare/4.18.1...4.19.2)

Updates `ws` from 8.14.2 to 8.17.1
- [Release notes](https://github.com/websockets/ws/releases)
- [Commits](https://github.com/websockets/ws/compare/8.14.2...8.17.1)

---
updated-dependencies:
- dependency-name: braces
  dependency-type: indirect
  dependency-group: npm_and_yarn
- dependency-name: express
  dependency-type: indirect
  dependency-group: npm_and_yarn
- dependency-name: ws
  dependency-type: indirect
  dependency-group: npm_and_yarn
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* fix: cwd should use workspaceFolder (#5210)

Problem:
- amazonq/toolkit fail to build on windows with: "The terminal process
  failed to launch: Starting directory (cwd) "\..\core" does not exist.
  inside of their tasks"

Solution:
- Use cwd that is relative to the workspace folder

* fix: activateExtension is used before globals is registered (#5211)

Problem
- Unit test setup is using activateExtension before globals is registered when running extension tests (current file)

Solution
- Instead of activateExtension just use vscode.extensions.getExtension directly
- Similiar problem was fixed in integ/e2e in https://github.com/aws/aws-toolkit-vscode/pull/5078

* chore: Add core outFiles to amazon q launch configs (#5212)

Problem:
- We didn't have the outFiles set in some q launch configs

Solution:
- Set them so we can use breakpoints

* auth: TimeoutError OIDC client fix attempt (#5213)

During API calls with the OIDC client the request would
time out and we would get TimeoutError.

Solution:

See if adding an explicit request timeout amount will avoid this

Signed-off-by: Nikolas Komonen <[email protected]>

* fix(ci): feature branch name missing from toolkit artifact #5222

Problem:
Because of the `npm run restorePackageJson` step, the `--feature separate-sessions`
arg is not passed to `package.ts`:

    npm run copyPackageJson && ts-node ../../scripts/package.ts && npm run restorePackageJson --feature separate-sessions

Thus the feature branch name is not in the toolkit artifact name. For
example, the feature/separate-sessions branch artifacts are:

    amazon-q-vscode-1.10.0-separate-sessions-g8a563a7.vsix
    aws-toolkit-vscode-3.11.0-g8a563a7.vsix

Solution:
Inline the `--restore` logic into `package.ts`. This is hacky, but
should be temporary after we resolved IDE-12831 so that these hacks are
no longer needed.

* fix: pass workspace folder to amazon q e2e tests (#5216)

Problem:
- workspace folder does not open when running testE2E for amazon q through command line

Solution:
- open the workspace folder

* cleanup: moved a bunch of static text into types and enums (#5225)

Co-authored-by: eugene yevhen andruszczenko <[email protected]>

* feat(policyCheck): CheckAccessNotGranted, CheckNoPublicAccess #5208

Problem
- IAM Access Analyzer recently released an update to custom policy checks adding
  support for public access and critical resources:
  https://aws.amazon.com/about-aws/whats-new/2024/06/aws-iam-access-analyzer-policy-checks/

Solution
- Support the updated checks.

References
- PR for adding telemetry for IamPolicyChecks:
  https://github.com/aws/aws-toolkit-common/pull/735
- PR for updating telemetry for updated checks:
  https://github.com/aws/aws-toolkit-common/pull/756
- PR for release of the IamPolicyChecks feature in toolkits:
  https://github.com/aws/aws-toolkit-vscode/pull/5028

* telemetry: scrub reasonDesc field #5215

followup to f010123ef2dd

* feat: update sendFeatureDevTelemetry to include user context and opt in/out telemetry options (#5221)

* tests: skip promptForConnection on "min vscode" CI #5228

Problem:
Test constantly fails on "minimum vscode" CI job:

    Auth
      promptForConnection
        reauthenticates a connection if the user selects an expired one:
     AssertionError [ERR_ASSERTION]: The expression evaluated to a falsy value:
     assert_1.default.ok(!refreshedConnItem.description?.match(/expired/i))
     + expected - actual

     -false
     +true
     at /codebuild/output/src3389505232/src/github.com/aws/aws-toolkit-vscode/packages/core/src/test/credentials/auth.test.ts:512:24
     at runMicrotasks (<anonymous>)
     at processTicksAndRejections (node:internal/process/task_queues:96:5)
     at async /codebuild/output/src3389505232/src/github.com/aws/aws-toolkit-vscode/packages/core/src/test/shared/vscode/window.ts:496:29

Solution:
Skip the test on the "minimum vscode" CI job.

* fix(featureDev): progress UX during backend calls (#5223)

* fix(featureDev): fixed in progress UX during backend calls

Problem:
- There was a mynah update which caused the in progress message to be hidden leading to bad experience for customers as they would not be aware that their calls are in progress.

Solution:
- Mynah changes were fixed with this PR: https://github.com/aws/mynah-ui/pull/67
- Bumped mynah-version to use latest fix

Co-authored-by: Dogus Atasoy <[email protected]>

* ci: fix windows global activation (#5154)

* fix: windows ci

Problem:
- Windows ci is no longer working after we moved amazonq tests to the
  amazonq package

Solution:
- Utilize upper case drive letters for getting the root in
  amazonq/toolkit. That way imports will resolve correctly.

Additional info:
In multi root npm workspaces on windows it looks like imports into other npm workspace packages makes the loaded module id an uppercase drive letter in the node require cache.

E.g. when we import a file from core the module ids inside of amazonq/toolkits node require cache are something like:
- C:\${pathToWorkspace}\packages\core\myfile.js

However, internal workspace package imports are lower case drive letters. That means when core imports a module inside of core we see this as:
- c:\${pathToWorkspace}\packages\core\myfile.js

This can cause things like globals to be undefined, since tests inside of amazonq/toolkit are looking for upper case module ids, whereas tests inside of core are always looking
for lower case module ids (since the tests live inside of core itself)

* fix: step function language server activation (#5226)

* fix: step function language server activation

Problem:
- After our 3.0 release step functions support no longer gets activated because of the harsher constraint we put on globals in https://github.com/aws/aws-toolkit-vscode/commit/fa54d7c0a07a4acb8675ffe89178a1c8c146cbd5

Solution:
- Remove references to global.clock since they aren't needed in this context

* docs: various #5230

* tests: unreliable test "submitFeedbackListener" #5232

Problem:
unreliable test:

    submitFeedbackListener
      submits feedback for Amazon Q, disposes, and handles errors:

     AssertionError [ERR_ASSERTION]: The expression evaluated to a falsy value:
     assert.ok(postStub.calledOnceWithExactly({ comment: comment, sentiment: sentiment }))
     + expected - actual
     -false
     +true
     at Context.<anonymous> (src/test/feedback/commands/submitFeedbackListener.test.ts:33:20)

This is because of a race. CodeTransform tests set a global "jobId",
which may be used by the feedback form, which causes unexpected results
in the feedback tests.

Solution:
Remove random special-case for CodeTransform, from `FeedbackWebview`.

* fix(threat composer): cleanup #5162

* Fetching the web view html if it's not available
* setting the file states in a reload action if it did not exist
* removing the default TC editor toggle, as it can be set using the VSCode Editor Associations setting
* Updating comments to reference the file state properties
* Call render when state is not 'loading'

* deps: update prettier, eslint, telemetry

* deps: remove redundant(?) deps from core/package.json

Not sure which things should go in the top-level package.json, but seems
like we should at least avoid "duplicates".

* lint

* deps: update marked, telemetry #5239

close https://github.com/aws/aws-toolkit-vscode/pull/5139

* Problem: Users who open vscode after previously editing an ASL document are not seeing error checking activate (#5234)

Solution: This fixes that behaviour where users will now see error checking activate when re-opening vscode and editing an ASL document from a previous session

Co-authored-by: Vaid Saraswat <[email protected]>

* Release 3.12.0

* Release 1.11.0

* Update version to snapshot version: 3.13.0-SNAPSHOT

* Update version to snapshot version: 1.12.0-SNAPSHOT

* fix(amazonq): prompt input, button styles #5246

Problem
- Button texts are getting cropped too short
- Button texts are too big
- Buttons don't show borders inside chat messages
- Prompt input becomes invisible if user types html special characters
- Prompt input and selected command are not horizontally aligned
- Prompt input doesn't show some lines if user pastes multiline content

Solution
- All the above issues addressed through mynah-ui updates.
[MynahUI 4.15.1 release notes](https://github.com/aws/mynah-ui/releases/tag/v4.15.1)
[MynahUI 4.15.0 release notes](https://github.com/aws/mynah-ui/releases/tag/v4.15.0)
[MynahUI 4.14.1 release notes](https://github.com/aws/mynah-ui/releases/tag/v4.14.1)

* fix(amazonq): typo #5249

* feat(amazonqFeatureDev): improve error messages #5231

Problem
We don't have granular error handling for some scenarios.

Solution
Update the messages.

* fix(Amazon Q Code Transform): UI messages, help links #5242

* tests: snooze tech debt reminder

* fix(gumby): do not check auth when initializing transformation hub #5252

Problem:
When VSCode is opened without having first been signed into `Amazon Q`
and the user attempts a code transformation, the Transformation Hub
window opens, but does not update, and Code Transform appears to hang.

Solution:
There is some initialization code that registers commands that other
parts of Gumby uses to update the Transformation Hub, but this code was
locked behind an auth check; if it fails, the commands are simply never
registered, even if the user signs in again later.

This portion of the code is not necessary to lock behind an auth check
(it doesn't save memory or make anything run faster to not have it run),
so the check has been removed.

Testing:
1. User opens VSCode without already being signed in; the user signs in and then transforms a project successfully
2. User opens VSCode without already being signed in; the user signs in and then transforms a project unsuccessfully (partial success)
3. User opens VSCode without already being signed in; the user signs in and then is not able to submit a project to be transformed (fails to build locally)
4. User opens VSCode without already being signed in; the user signs in and then transforms a project that triggers HIL
5. User opens VSCode already having been signed in; the user transforms a project successfully

* fix(amazonq auto scan): spurious yellow lines in editor #5237

Problem:
- If user triggers a scan and modifies the code in the editor, the security
  issue shows yellow squiggle lines at different place in the editor.

Solution:
- Compare the existing code diff from editor with the codeSnippet from response.
  If match, show the issue to the user, else drop the suggestion.
- Implemented only for Auto-Scans.

* web: Fix failing web mode due to `fs-extra` import (#5248)

* docs: Update web mode docs

- Add information about a common error so that developers can hopefully ctrl+f this when
  debugging
- Remove the more complex image of dependency cruiser as it just causes confusion

Signed-off-by: Nikolas Komonen <[email protected]>

* web: make 'common' codewhisperer export module

Problem:

When importing the codewhisperer exports module from index.ts in web mode
it was breaking due to a transitive fs-extra import.

Solution:

Export a separate module from indexCommon.ts for code that works in web
or node. This file is a subset of index.ts, so behavior remains the same.

Signed-off-by: Nikolas Komonen <[email protected]>

---------

Signed-off-by: Nikolas Komonen <[email protected]>

* telemetry: set more MetricBase fields #5255

Problem:
`telemetry.run()` and `runCommand` do not set some "standard" fields on
failure.

Solution:
Update `telemetry.run()` and `runCommand`.

ref https://github.com/aws/aws-toolkit-common/pull/769

* fix(codecatalyst): simplify InactivityMessage #5260

Problem:
- The "progress" message tests are flaky because of the complex logic, and sometimes fail:
  ```
    1) InactivityMessages
         shows expected messages 5 minutes before shutdown on a 60 minute inactivity timeout:
       AssertionError [ERR_ASSERTION]: Expected 3 messages, but got 2
        at assertMessagesShown (src/testInteg/codecatalyst/devEnv.test.ts:182:20)
        at async Context.<anonymous> (src/testInteg/codecatalyst/devEnv.test.ts:85:9)
  ```
- Toolkit has complex logic for showing a "progress" message when the
  codecatalyst dev env timeout is approaching, followed by different
  logic for showing the "final" message. This isn't worth the complexity
  because if the user sees the message, clicking it is zero-cost and
  will happen immediately, otherwise the user isn't active and won't see
  any of the messages anyway.
- The non-modal progress message may be *hidden* if the user enabled
  vscode's "do not disturb" feature.

Solution:
- Remove the non-modal "progress" message. Only show the final, modal
  message **starting 5 min before shutdown**. Display it until user
  clicks it.

Followup to https://taskei.amazon.dev/tasks/IDE-13892

* refactor: Move aws services in to `src/awsService/` (#5253)

* refactor: start moving services to `awsService`

Problem:

We have multiple aws services that each have their own folder in the root.
This adds more folder to have to search through in the top level

Solution:

Create an `awsService` folder and start the move of the existing folders
in to them. This commit moves CDK for now, and in future commits we will move
the rest.

Signed-off-by: Nikolas Komonen <[email protected]>

* refactor: move cloudWatchLogs in to awsService

Signed-off-by: Nikolas Komonen <[email protected]>

* refactor: ec2 in to awsService/

Signed-off-by: Nikolas Komonen <[email protected]>

* refactor: ecr -> awsService/

Signed-off-by: Nikolas Komonen <[email protected]>

* refactor: apiGateway -> awsService/

Signed-off-by: Nikolas Komonen <[email protected]>

* refactor: ecs -> awsService/

Signed-off-by: Nikolas Komonen <[email protected]>

* refactor: iot -> awsService/

Signed-off-by: Nikolas Komonen <[email protected]>

* refactor: redshift -> awsService/

Signed-off-by: Nikolas Komonen <[email protected]>

* refactor: s3 -> awsService/

Signed-off-by: Nikolas Komonen <[email protected]>

* refactor: apprunner -> awsService/

Signed-off-by: Nikolas Komonen <[email protected]>

* refactor: accessanalyzer -> awsService/

Signed-off-by: Nikolas Komonen <[email protected]>

---------

Signed-off-by: Nikolas Komonen <[email protected]>

* web: minor Timer/Timeout change for CW (#5258)

* web: change timeout for web compatiblility

Problem:

Timer in this class does not work in web mode since it only
works with the Node timeout class.

Solution:

Use the generalized Timeout class.

Signed-off-by: Nikolas Komonen <[email protected]>

* test: copy timeout tests to web tests

This is copying the node timeout tests to web mode. Since they behave the same,
no modifications needed to be done to the tests.

- Also fix an issue where some code was not being run in web mode during refresh.

Signed-off-by: Nikolas Komonen <[email protected]>

---------

Signed-off-by: Nikolas Komonen <[email protected]>

* feat(telemetry): emit on opt-out and opt-in (#5259)

- Emits 1 final metric on opt-out that will let us know the user opted out.
- Also, on opt out it will emit any telemetry that was previously recorded and not sent.

* fix: restorePackageJson failure after stopping toolkit extension (#5262)

Problem:
- When debugging toolkit the tasks rely on the restorePackageJson script that was removed in https://github.com/aws/aws-toolkit-vscode/pull/5222

Solution:
- Re-add the restorePackageJson script

* fix: race condition in step functions language server activation test (#5244)

### Problem:
Sometimes the language server can start and fire the onReady event before we are listening for it in the tests, causing the tests to never receive the event

### Solution:
Set isReady on the ASLLanguageClient class and wait for the client to be ready in the test

* feat(amazonq-chat): customization support #5254

Problem:
Chat API: generateAssistantResponse doesn't pass customizationArn.

Solution:
- add customizationArn if applicable when the API is called
- move customization tree node from "inline suggestion" to "other Q features"
- send customizationArn if applicable when sending addMessage telemetry event

* fix(ci): noisy lines in Linux CI output #5268

Problem:

    [3422:0705/221403.718606:ERROR:bus.cc(407)] Failed to connect to the bus: Failed to connect to socket /run/dbus/system_bus_socket: No such file or directory
    [3422:0705/221403.916070:ERROR:bus.cc(407)] Failed to connect to the bus: Could not parse server address: Unknown address type (examples of valid types are "tcp" and on UNIX "unix")
    [3449:0705/221404.002987:ERROR:viz_main_impl.cc(196)] Exiting GPU process due to errors during initialization
    [3519:0705/221404.270770:ERROR:command_buffer_proxy_impl.cc(131)] ContextResult::kTransientFailure: Failed to send GpuControl.CreateCommandBuffer.

Solution:
- These lines are related to `xvfb` and are irrelevant. See https://github.com/cypress-io/cypress/issues/19299
- Drop the lines when printing output in Linux CI (AWS CodeBuild).

* fix(fs): FileSystemCommon lacks error handling

Note: This commit is part of a series.

Problem:
- Our `fs.ts` module does not have the advanced error-handling present
  in our `SystemUtilities` module.
- `fsCommon.delete()` defaults to `recursive:true`, which means
  directories may be unintentionally deleted when a file was expected,
  and related errors are not surfaced.

Solution:
- Merge the features of `SystemUtilities` into `fs.ts`.
- Migrate `SystemUtilities.delete()`.
- Change `fsCommon.delete()` to default to `recursive:false`.
  - Update callers to explicitly pass `recursive:true`, where appropriate.

* refactor(fs): FileSystem.exists()

Note: This commit is part of a series.

Problem:
- Error handling is inconsistent.
- `SystemUtilities` is redundant with `fs.ts`, is not web-compatible.

Solution:
- Merge the features of `SystemUtilities` into `fs.ts`.
- Migrate tests.

* refactor(fs): FileSystem.readFile()

Note: This commit is part of a series.

Problem:
- Error handling is inconsistent.
- `SystemUtilities` is redundant with `fs.ts`, and is not web-compatible.

Solution:
- Merge the features of `SystemUtilities` into `fs.ts`.
- Migrate tests.

* refactor(fs): FileSystem.writeFile()

Note: This commit is part of a series.

Problem:
- Token files in ~/.aws/sso/cache/ are readable by all users on the system. IDE-10618
- `SystemUtilities` is redundant with `fs.ts`, and is not web-compatible.

Solution:
- When `opt` is provided to `writeFile()`, use the nodejs filesystem
  interface instead of vscode VFS. Don't know of any other way to allow
  callers to specify the permissions/flags.
- Merge the features of `SystemUtilities` into `fs.ts`.
- Migrate tests.

* refactor(fs): FileSystem.checkPerms()

* feat(logging): log(logLevel, …)

* fix: validate path for getHomeDirectory

* fix(env): fs.getUserHomeDir()

Problem:
- `SystemUtilities.getHomeDirectory`:
  - does not use the cross-platform `fs.ts` module.
  - checks env vars every time it is called, which is a performance cost.

Solution:
- Move validation into a one-time `initUserHomeDir()` function which is
  called on startup.
- Introduce `fs.getUserHomeDir()`.

* refactor(fs): pathFind.ts #5273

Problem:
SystemUtilities is deprecated and redundant.

Solution:
Move remaining logic into new `pathFind.ts` module.

* fix: ReferenceError: Node is not defined in e2e tests (#5275)

Problem:
- We are observing "ReferenceError: Node is not defined" in the e2e test logs

Solution:
- Add Node into the global object

* feat(feature dev): exclude non-code files in "prepare" step #5238

Problem:
Feature dev compiles a compressed file with relevant local files to prepare
response. This includes some irrelevant files.

Current approach: exclude gitignores and big files (>1024000 kb).

Solution:
Allowlist specific specific extensions, drop all others.

Follow up: emit telemetry event, once it's defined.

* fix(appcomposer): add warning message to version 1.91.0 about drag an… (#5278)

* fix(appcomposer): add warning message to version 1.91.0 about drag and drop

Problem:
VS Code version 1.91.0 has a bug causing the dragEnd action to not trigger in WebViews unless Shift is held down.

Solution:
This has already been reverted and should be fixed in the next VS Code version. In the meantime, this adds a warning message informing users of the workaround.

* update wording

* update wording again

---------

Co-authored-by: Jacob Largent <[email protected]>

* Release 1.12.0

* Update version to snapshot version: 1.13.0-SNAPSHOT

* Release 3.13.0

* Update version to snapshot version: 3.14.0-SNAPSHOT

* refactor(fs): eliminate srcShared/ namespace

Problem:
The `srcShared/` concept is not necessary. Instead, a more tractable and
less-churn approach is:
1. Assume that ALL modules are "cross platform" (i.e. they support both
   nodejs and web-mode).
2. Web-only logic lives in a `web/` directory relative to its namespace
   (or in some rare cases, a `fooWeb.ts` module).
2. Nodejs-only logic lives in a `nodejs/` directory relative to its namespace
   (or in some rare cases, a `fooNodejs.ts` module).

See also:
- https://github.com/aws/aws-toolkit-vscode/pull/5046#discussion_r1614541013
- https://github.com/aws/aws-toolkit-vscode/pull/5248#discussion_r1661084971

Solution:
- Move `fs.ts` into `shared/`.
- Delete `srcShared/`.

* refactor(fs): eliminate SystemUtilities

* refactor(fs): rename "fsCommon"

* fix(env): getUserInfo() may fail on AD/LDAP #5279

Problem:
Both AWS Toolkit and Amazon Q fail to start when the effective user
account is defined externally (e.g. by AD/LDAP). #5277

    [error] SystemError: A system error occurred: uv_os_get_passwd returned ENOENT (no such file or directory)

Bisected to #5215 which calls os.userInfo in getUsername (env.ts).

When the effective user ID does not have a matching entry in
/etc/passwd, userInfo throws an exception. Discussed in:
- https://github.com/microsoft/vscode-remote-release/issues/9649
- https://github.com/cyjake/ssh-config/issues/77

Steps to reproduce:
1. Login as a user that is synced to AD/LDAP.
2. Install AWS Toolkit and Amazon Q
3. Errors in `~/.config/Code/logs/<date and time>/window1/exthost/exthost.log`
   ```
   2024-07-08 12:09:54.179 [error] Activating extension amazonwebservices.amazon-q-vscode failed due to an error:
   2024-07-08 12:09:54.179 [error] SystemError: A system error occurred: uv_os_get_passwd returned ENOENT (no such file or directory)
           at new SystemError (node:internal/errors:257:5)
           at new NodeError (node:internal/errors:368:7)
           at Object.userInfo (node:os:365:11)
   ```

Solution:
- Move getUserInfo/getUsername into fs.ts since they depend on the
  filesystem.
- If userInfo fails, fallback to (in order of precedence):
  - process.env.USER
  - getUserHomeDir() directory name
  - "unknown-user"

* changelog

* fix: remove auth screen background

* refactor(test): move samples/ into testFixtures/ #5285

Problem:
Some test fixtures live outside of testFixtures/.

Solution:
Move them into testFixtures/.

* fix: code coverage upload fails (#5286)

## Problem
If you don't set a code coverage token it uses a global shared codecoverage token that can be rate limited

## Solution
Switch to our codecoverage token

* fix: eslint DeprecationWarning #5287


    (node:99880) DeprecationWarning: "no-incorrect-once-usage" rule is
    using `context.getScope()`, which is deprecated and will be removed
    in ESLint v9. Please use `sourceCode.getScope()` instead. (Use `node
    --trace-deprecation ...` to show where the warning was created)

Notes:
Consider:
```
L1: const x = 1
L2: const y = 2, z = 3
```
There are two `VariableDeclaration` here (L1, L2). L1 node.declarations is `[x=1]`. L2 node.declarations is `[y=2, z=3]`.

Whether you take the root node level (L1/L2) or the individual declarations in each root, the scope is the same.

* feat(amazonq): Chat Project Context With LSP #5271

Problem:
Amazon Q chat is not aware of files opened in user's workspace. User
should be able to ask Q any question regarding current open workspace.

Solution:
Perform local indexing with CodeSage model in LSP. Get the most relevant
files for user chat input, and use that input to add project-scoped
context when user asks Q any question regarding current open workspace.

1. Download and installation of LSP server, follow DEXP download
   manifest. We enforce the LSP version in the IDE so that when we
   release new LSP, there is no production impact. A new extension is
   required to run new LSP.
2. Start the LSP server with the stdin encryption spec from DEXP.
3. Call the LSP server to index project,
4. Call the LSP server to retrieve project context
5. New settings for the LSP server. Enable/Disable, Threads, Use GPU or not.
6. New metrics.

* fix(access analyzer): UI is blank #5291

Problem:
After 198d2c9946adfd2a0f0c4ff567a8da2a985c72e6 moved AWS services to the
`awsService` folder, this broke Access Analyzer integration, since `sourcePath`
was hardcoded to `src/accessanalyzer/vue/index.js`. The integration UI is blank.

Solution:
Fix the hardcoded path.
The integration UI shows up after the fix.

* lint: disallow executeCommand('setContext')

* feat(setContext): hoist setContext into a shared module

Problem:
The vscode 'setContext' command:
- is not type-checked
- usages cannot be found programmatically (i.e. via "find references")
- keys are not centralized, so it's not clear which keys we are setting

Solution:
- Introduce a `setContext` module. Define keys as a `type`.
- Add a lint rule which prevents direct use of the vscode command.

* refactor: globalState abstraction

* Update Feature-69f0d3bf-3e57-4ad6-93ea-48de3c890331.json

* Release 1.13.0

* Update version to snapshot version: 1.14.0-SNAPSHOT

* feat: proactively show code generation iterations (#5282)

Problem:
- Currently, there is no information shown to customers in the chat regarding the number of iterations they could run during code generation.
- They only get to know this on hitting the iteration limits and seeing the error like you have reached the limit for number of iterations.

Solution:
- Hence, with this change, we are showing the remaining iterations proactively by exposing the remaining iteration numbers and total iteration numbers to the end of each code generation.

* feat: Add codecov to integ/e2e/linux tests (#5295)

* feat: Add codecov to integ/e2e/linux tests

Problem:
- We don't have code coverage reporting for our integ/e2e/linux tests

Solution:
- Add it

* fix: refactor broke webview paths (#5299)

Problem:

A recent refactor did not update certain static paths which prevent
webviews from working.

Solution:

Fix the incorrect paths.

Signed-off-by: Nikolas Komonen <[email protected]>

* fix: app composer generate suggestions unclickable when logged in (#5256)

Problem:
- app composers generate suggestions became unclickable when we switched back to seperate auth since they were relying on auth session sharing

Solution:
- allow extensions to call into auth/chat for amazon q. This allows toolkit to have features that depend on q auth without exposing all of auth itself

* Release 1.14.0

* Release 3.14.0

* fix: font color for high contrast themes

* Update version to snapshot version: 1.15.0-SNAPSHOT

* Update version to snapshot version: 3.15.0-SNAPSHOT

* feat: globalState abstraction

Problem:
We have a settings.ts module which abstracts the vscode settings
interface, to gain important features such as type-checking, validation,
and error handling.
https://github.com/aws/aws-toolkit-vscode/blob/b9d5534c0879382baa5c900d14d0a2c3fdd529c5/packages/core/src/shared/settings.ts

The vscode globalState interface is very similar to the vscode settings
interface, and has the same limitations and potential for bugs: the data
is user-defined and arbitrary, thus the types are unknown and must
always be runtime-checked, which is a verbose and often overlooked step.

Examples:

- `redshiftState.ts` https://github.com/aws/aws-toolkit-vscode-staging/pull/1034/files
- https://github.com/aws/aws-toolkit-vscode/blob/8f55e40cab47ef7d25ed5faac274c3fac3f9f91c/src/shared/filesystemUtilities.ts#L228-L253
- globalState type issues related to a codewhisperer bug: https://github.com/aws/aws-toolkit-vscode/pull/3060
- `aws.lastUploadedToS3Folder` https://github.com/aws/aws-toolkit-vscode/pull/3183/files
- `ExtensionUse` class https://github.com/aws/aws-toolkit-vscode/pull/3634/files
- `codewhisperer/util/globalStateUtil.test.ts` https://github.com/aws/aws-toolkit-vscode/blob/80e715bbf3e6eb354a9b6e5e327c732b89df38e3/packages/amazonq/test/unit/codewhisperer/util/globalStateUtil.test.ts

Solution:
- Introduce a `globalState` wrapper, similar to `src/shared/settings.ts`.
- Migrate the `redshiftState.ts` module into the centralized `globalState`
  module.

* fix(ssoAccessTokenProvider): use globalState abstraction

* fix(amazonq): visual bugs in prompt input field #5308

## Problem
There are 2 minor visual bugs in Amazon Q Chat UI:
* When the prompt input text exceeds the width of the chat box it's not always wrapped correctly.
* When user types empty spaces at the beginning of lines inside prompt, it appears like a code block instead of a paragraph.

## Solution
* Prompt input is properly sized and matches with the textarea underneath.
* Removed empty spaces for each line.

[MynahUI 4.15.2](https://github.com/aws/mynah-ui/releases/tag/v4.15.2)

* lint: disallow console.log and similar

* lint: fix or ignore uses of console.log

* fix(ci): codecov throttled on GHA #5310

Problem:

    [2024-07-12T13:58:12.874Z] ['info'] -> No token specified or token is empty
    ...
    [2024-07-12T13:58:13.715Z] ['verbose'] Passed token was 0 characters long
    [2024-07-12T13:58:13.952Z] ['verbose'] The error stack is: Error:
    Error uploading to https://codecov.io: Error: There was an error
    fetching the storage URL during POST: 429 - {'detail':
    ErrorDetail(string='Rate limit reached. Please upload with the
    Codecov repository upload token to resolve issue. Expected time to
    availability: 1752s.', code='throttled')}

Solution:
- Update codecov GHA runner.
- Set "token" field of the codecov GHA runner.

* lint: set prettier arrowParens=always

Problem:
javascript arrow functions allow braces `{ ... }` to surround an
expression without a `return`, which leads to bugs such as
https://github.com/aws/aws-toolkit-vscode/pull/3659
https://github.com/aws/aws-toolkit-vscode/issues/3662

Example:

    return supplementalContexts.filter(item => { item.content.trim().length !== 0 })

should be:

    return supplementalContexts.filter(item => item.content.trim().length !== 0)

Solution:
- Set prettier `arrowParens=always`. This is the default, explained
  here: https://prettier.io/docs/en/options.html#arrow-function-parentheses
  > At first glance, avoiding parentheses may look like a better choice
  because of less visual noise. However, when Prettier removes
  parentheses, it becomes harder to add type annotations, extra
  arguments or default values as well as making other changes.
  Consistent use of parentheses provides a better developer experience
  when editing real codebases, which justifies the default value for the
  option.
- Note: this is equivalent to the eslint `@stylistic/no-confusing-arrow` rule.
  - https://eslint.style/packages/default#stylistic-eslint-plugin
  - https://eslint.style/rules/default/no-confusing-arrow

* lint: prettier decides eslint

https://prettier.io/docs/en/integrating-with-linters.html

* build: remove "format" task

This task was added as workaround. It's not need now because we have
"eslint-plugin-prettier" installed which tells eslint to include rules
decided by our prettier config.

* build: remove "formatfix" task

Since our eslint rules now are informed by our prettier config
(performed by the "eslint-plugin-prettier" package), `prettier --write`
is unnecessary because it's implictly done by `eslint --fix`.

https://github.com/prettier/eslint-plugin-prettier

* lint: use .gitignore to decide eslint ignore patterns

TODO: extract the eslint invocation to scripts/lint.ts so that it's not
duplicated in the "lintfix" task.

* lint: run lint on all projects

---------

Signed-off-by: Nikolas Komonen <[email protected]>
Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: aws-toolkit-automation <[email protected]>
Co-authored-by: Vikash Agrawal <[email protected]>
Co-authored-by: Justin M. Keyes <[email protected]>
Co-authored-by: Josh Pinkney <[email protected]>
Co-authored-by: aws-toolkit-automation <>
Co-authored-by: Nikolas Komonen <[email protected]>
Co-authored-by: Maxim Hayes <[email protected]>
Co-authored-by: Lei Gao <[email protected]>
Co-authored-by: David <[email protected]>
Co-authored-by: Maxim Hayes <[email protected]>
Co-authored-by: Nick Ardecky <[email protected]>
Co-authored-by: Laxman Reddy <[email protected]>
Co-authored-by: JuliaABurch <[email protected]>
Co-authored-by: chengoramazon <[email protected]>
Co-authored-by: invictus <[email protected]>
Co-authored-by: Diana Abitova <[email protected]>
Co-authored-by: Santiago Martin <[email protected]>
Co-authored-by: Dogus Atasoy <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Eugene Andruszczenko <[email protected]>
Co-authored-by: eugene yevhen andruszczenko <[email protected]>
Co-authored-by: alankuo-aws <[email protected]>
Co-authored-by: kumsmrit <[email protected]>
Co-authored-by: Dogus Atasoy <[email protected]>
Co-authored-by: Bijin Abraham Idicula <[email protected]>
Co-authored-by: VaidSaraswat <[email protected]>
Co-authored-by: Vaid Saraswat <[email protected]>
Co-authored-by: Will Lo <[email protected]>
Co-authored-by: Thiago Verney <[email protected]>
Co-authored-by: Ting Cheng <[email protected]>
Co-authored-by: wilson <[email protected]>
Co-authored-by: JLargent <[email protected]>
Co-authored-by: Jacob Largent <[email protected]>
Co-authored-by: linyuxi0511 <[email protected]>
Co-authored-by: Viktor Shcherba <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants