Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Build Broken on Swift for Tensorflow Stock Toolchain Nightly Builds #142

Closed
xanderdunn opened this issue Jan 29, 2021 · 4 comments · Fixed by #143
Closed

Build Broken on Swift for Tensorflow Stock Toolchain Nightly Builds #142

xanderdunn opened this issue Jan 29, 2021 · 4 comments · Fixed by #143

Comments

@xanderdunn
Copy link

xanderdunn commented Jan 29, 2021 •
edited
Loading 

$ which swift
/home/xander/swift-tensorflow-DEVELOPMENT-cuda10.2-cudnn7-stock-ubuntu18.04/usr/bin/swift
$ swift --version
Swift version 5.3-dev (LLVM a80ce7032652e19, Swift 6a6f972f7fcf51d)
Target: x86_64-unknown-linux-gnu
$ git log
commit fca2fc0e0581656449f1cee40ec5513f7a957446 (HEAD -> main, origin/main, origin/HEAD)
Author: Dave Abrahams <[email protected]>
Date:   Mon Dec 21 13:39:50 2020 -0800

    Make base of Projections public. (#138)
[4/4] Compiling PenguinTables TypedColumn.swift
$ swift package clean
$ swift build
createBuildSystem false
/home/xander/dev/penguin/Sources/PenguinParallel/ConcurrencyPlatform.swift:105:5: warning: future versions of Swift reserve the word 'await'; if this name is unavoidable, use backticks to escape it
    await(predicate)
    ^~~~~
    `await`
/home/xander/dev/penguin/Sources/PenguinParallel/ConcurrencyPlatform.swift:105:5: warning: future versions of Swift reserve the word 'await'; if this name is unavoidable, use backticks to escape it
    await(predicate)
    ^~~~~
    `await`
/home/xander/dev/penguin/Sources/PenguinTables/TableSchema.swift:20:33: error: cannot find type 'KeyPathIterable' in scope
  public protocol PTableSchema: KeyPathIterable, PDefaultInit {
                                ^~~~~~~~~~~~~~~
/home/xander/dev/penguin/Sources/PenguinTables/TableSchema.swift:26:27: error: value of type 'Self' has no member 'allKeyPaths'
      let keyPaths = self.allKeyPaths as! [PartialKeyPath<Self>]
                     ~~~~ ^~~~~~~~~~~
/home/xander/dev/penguin/Sources/PenguinTables/TableSchema.swift:20:33: error: cannot find type 'KeyPathIterable' in scope
  public protocol PTableSchema: KeyPathIterable, PDefaultInit {
                                ^~~~~~~~~~~~~~~
/home/xander/dev/penguin/Sources/PenguinTables/TableSchema.swift:26:27: error: value of type 'Self' has no member 'allKeyPaths'
      let keyPaths = self.allKeyPaths as! [PartialKeyPath<Self>]
                     ~~~~ ^~~~~~~~~~~
/home/xander/dev/penguin/Sources/PenguinTables/TableSchema.swift:20:33: error: cannot find type 'KeyPathIterable' in scope
  public protocol PTableSchema: KeyPathIterable, PDefaultInit {
                                ^~~~~~~~~~~~~~~
/home/xander/dev/penguin/Sources/PenguinTables/TableSchema.swift:20:33: error: cannot find type 'KeyPathIterable' in scope
  public protocol PTableSchema: KeyPathIterable, PDefaultInit {
                                ^~~~~~~~~~~~~~~

I just tried this using the latest nightly as of 29 January 2021 and got the same result.
@saeta
Copy link
Owner

saeta commented Jan 29, 2021

Hi @xanderdunn !

I've tried to reproduce this issue, and unfortunately I was unsuccessful. I started up a fresh GCE VM to ensure I had a clean environment, and installed the latest nightly for Ubuntu 20.04. The following is an abbreviated console traceback. Could you perhaps provide more information to help me reproduce the error?

Thanks!
-Brennan

Welcome to Ubuntu 20.04.1 LTS (GNU/Linux 5.4.0-1034-gcp x86_64)
 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage
  System information as of Fri Jan 29 19:30:00 UTC 2021
  System load:  0.99              Processes:             128
  Usage of /:   2.8% of 48.29GB   Users logged in:       0
  Memory usage: 7%                IPv4 address for ens4: 10.138.0.62
  Swap usage:   0%
1 update can be installed immediately.
0 of these updates are security updates.
saeta_google_com@saeta-test:~$ wget https://storage.googleapis.com/swift-tensorflow-artifacts/releases/v0.12/rc2/sw
ift-tensorflow-RELEASE-0.12-ubuntu20.04.tar.gz
--2021-01-29 19:31:24--  https://storage.googleapis.com/swift-tensorflow-artifacts/releases/v0.12/rc2/swift-tensorf
low-RELEASE-0.12-ubuntu20.04.tar.gz
Resolving storage.googleapis.com (storage.googleapis.com)... 172.253.117.128, 74.125.195.128, 74.125.142.128, ...
Connecting to storage.googleapis.com (storage.googleapis.com)|172.253.117.128|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 556783713 (531M) [application/octet-stream]
Saving to: ‘swift-tensorflow-RELEASE-0.12-ubuntu20.04.tar.gz’
swift-tensorflow-RELEASE-0.1 100%[=============================================>] 530.99M  91.9MB/s    in 5.9s    
2021-01-29 19:31:31 (89.3 MB/s) - ‘swift-tensorflow-RELEASE-0.12-ubuntu20.04.tar.gz’ saved [556783713/556783713]
saeta_google_com@saeta-test:~$ tar -zxvf swift-tensorflow-RELEASE-0.12-ubuntu20.04.tar.gz 
usr/
usr/lib/
usr/lib/libIndexStore.so
usr/lib/swift_static/
usr/lib/swift_static/shims/
usr/lib/swift_static/shims/KeyPath.h
usr/lib/swift_static/shims/ThreadLocalStorage.h
usr/lib/swift_static/shims/NSIndexSetShims.h
usr/lib/swift_static/shims/NSLocaleShims.h
usr/lib/swift_static/shims/DispatchOverlayShims.h
usr/lib/swift_static/shims/LibcOverlayShims.h
[[ ... snip ... ]]
usr/local/include/indexstore/IndexStoreCXX.h
usr/local/include/indexstore/indexstore.h
saeta_google_com@saeta-test:~$ sudo apt-get update
Hit:1 http://us-west1.gce.archive.ubuntu.com/ubuntu focal InRelease
Get:2 http://us-west1.gce.archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB]                        
Get:3 http://us-west1.gce.archive.ubuntu.com/ubuntu focal-backports InRelease [101 kB]                      
Get:4 http://us-west1.gce.archive.ubuntu.com/ubuntu focal/universe amd64 Packages [8628 kB]                 
Get:5 http://security.ubuntu.com/ubuntu focal-security InRelease [109 kB]
[[ ... snip ... ]]

Reading package lists... Done
saeta_google_com@saeta-test:~$
saeta_google_com@saeta-test:~$ sudo apt-get install clang libpython2-dev libblocksruntime-dev
Reading package lists... Done
Building dependency tree       
Reading state information... Done
[[ ... snip ... ]]

Processing triggers for install-info (6.7.0.dfsg.2-5) ...
saeta_google_com@saeta-test:~$
saeta_google_com@saeta-test:~$ cd usr/bin/
saeta_google_com@saeta-test:~/usr/bin$ echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
saeta_google_com@saeta-test:~/usr/bin$ export PATH=`pwd`:$PATH
saeta_google_com@saeta-test:~/usr/bin$ cd
saeta_google_com@saeta-test:~$ git clone https://github.com/saeta/penguin.git
Cloning into 'penguin'...
remote: Enumerating objects: 473, done.
remote: Counting objects: 100% (473/473), done.
remote: Compressing objects: 100% (266/266), done.
remote: Total 7361 (delta 308), reused 299 (delta 186), pack-reused 6888
Receiving objects: 100% (7361/7361), 10.63 MiB | 24.20 MiB/s, done.
Resolving deltas: 100% (5597/5597), done.
saeta_google_com@saeta-test:~$ cd penguin/
saeta_google_com@saeta-test:~/penguin$ swift build
Fetching https://github.com/apple/swift-argument-parser
Fetching https://github.com/google/swift-benchmark.git
Cloning https://github.com/apple/swift-argument-parser
Resolving https://github.com/apple/swift-argument-parser at 0.3.1
Cloning https://github.com/google/swift-benchmark.git
Resolving https://github.com/google/swift-benchmark.git at 0.1.0
[182/182] Linking PenguinBenchmarks

* Build Completed!
saeta_google_com@saeta-test:~/penguin$ swift test
/home/saeta_google_com/penguin/Tests/PenguinStructuresTests/AnyArrayBufferTests.swift:69:27: warning: 'unsafelyMutate(assumingElementType:_:)' is deprecated: use subscript(unsafelyAssumingElementType:) instead.
    let baseAddress0 = a0.unsafelyMutate(assumingElementType: Type<Int>()) {
                          ^
/home/saeta_google_com/penguin/Tests/PenguinStructuresTests/AnyArrayBufferTests.swift:76:27: warning: 'unsafelyMutate(assumingElementType:_:)' is deprecated: use subscript(unsafelyAssumingElementType:) instead.
    let baseAddress1 = a0.unsafelyMutate(assumingElementType: Type<Int>()) {
                          ^
[[ ... snip ... ]]
[104/104] Linking PenguinPackageTests.xctest

* Build Completed!
Test Suite 'All tests' started at 2021-01-29 19:35:40.333
Test Suite 'debug.xctest' started at 2021-01-29 19:35:40.334
Test Suite 'BoolComparableTests' started at 2021-01-29 19:35:40.334
[[ ... snip ... ]]
Test Suite 'debug.xctest' passed at 2021-01-29 19:35:44.997
         Executed 379 tests, with 0 failures (0 unexpected) in 4.643 (4.643) seconds
Test Suite 'All tests' passed at 2021-01-29 19:35:44.997
         Executed 379 tests, with 0 failures (0 unexpected) in 4.643 (4.643) seconds
saeta_google_com@saeta-test:~/penguin$ git log --oneline | head
fca2fc0 Make base of Projections public. (#138)
41a64d0 Add update/updateAll (#139)
e837012 Update link to penguin tables (#136)
2ed8978 Regenerate all docs (#134)
738d7c8 workaround compile crash (#132)
61d4fce Point to swift-benchmark 0.1.0 (#133)
9044c02 Update apple/swift links -> main (#128)
88bc092 make swift-benchmark dependency use version (#130)
da4c484 Fix co-compilation with Swift-Models under 0.12 compiler (#129)
0e1d031 InsertionOrderedDictionary: Codable - Don't encode too much.
saeta_google_com@saeta-test:~/penguin$ swift --version
Swift version 5.3-dev (LLVM 69d8678431d3eee, Swift e1aef96b7fea59b)
Target: x86_64-unknown-linux-gnu
saeta_google_com@saeta-test:~/penguin$

@xanderdunn
Copy link
Author

xanderdunn commented Jan 29, 2021 •
edited
Loading

It looks like you're using the 0.12 release build there: wget https://storage.googleapis.com/swift-tensorflow-artifacts/releases/v0.12/rc2/swift-tensorflow-RELEASE-0.12-ubuntu20.04.tar.gz

I am also successfully building on 0.12 RELEASE right now. This is specific to the new stock toolchains. The nightly builds. See my top line:

$ which swift
/home/xander/swift-tensorflow-DEVELOPMENT-cuda10.2-cudnn7-stock-ubuntu18.04/usr/bin/swift

How does this work for you when you switch to the nightly toolchains?

I also happen to be on Ubuntu 18.04, but I doubt that's related.

The toolchain I'm getting:

$ wget https://storage.googleapis.com/swift-tensorflow-artifacts/nightlies/latest/swift-tensorflow-DEVELOPMENT-cuda11.0-cudnn8-stock-ubuntu18.04.tar.gz

@xanderdunn xanderdunn changed the title Build Broken on Swift for Tensorflow Nightly Builds Build Broken on Swift for Tensorflow Stock Toolchain Nightly Builds Jan 29, 2021
saeta added a commit that referenced this issue Jan 29, 2021
@saeta
Disable KeyPathIterable
c82dae4 
Previously, PenguinTable's TableSchema conditionally depended upon
`KeyPathIterable` to automatically implement some functionality.

After S4TF 0.12, S4TF moved to a different implementation of KeyPathIterable,
and as a result, KeyPathIterable is no longer included in the standard library.
This causes compilation errors in Penguin.

This change disables the conditional use of KeyPathIterable and future work can
refactor the implementation to use the new S4TF-contributed reflection APIs.

Fixes: #142.
@saeta saeta mentioned this issue Jan 29, 2021
Merged
@saeta
Copy link
Owner

saeta commented Jan 29, 2021

Argh; you're absolutely right. My mistake. Thank you for the extra information. Please see #143 for a quick-fix.

saeta added a commit that referenced this issue Jan 29, 2021
@saeta
Disable KeyPathIterable (#143)
3ad1568 
Previously, PenguinTable's TableSchema conditionally depended upon
`KeyPathIterable` to automatically implement some functionality.

After S4TF 0.12, S4TF moved to a different implementation of KeyPathIterable,
and as a result, KeyPathIterable is no longer included in the standard library.
This causes compilation errors in Penguin.

This change disables the conditional use of KeyPathIterable and future work can
refactor the implementation to use the new S4TF-contributed reflection APIs.

Fixes: #142.
@xanderdunn
Copy link
Author

Huge thanks @saeta, the build works for me now!

@ProfFan ProfFan mentioned this issue Dec 17, 2022
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Disable KeyPathIterable saeta/penguin
2 participants
@saeta @xanderdunn