Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TXT export: store data in memory and write only at last timestep #863

Open
RemDelaporteMathurin opened this issue Aug 2, 2024 · 4 comments · May be fixed by #883
Open

TXT export: store data in memory and write only at last timestep #863

RemDelaporteMathurin opened this issue Aug 2, 2024 · 4 comments · May be fixed by #883
Labels
enhancement New feature or request

Comments

@RemDelaporteMathurin
Copy link
Collaborator

@jhdark suggested that instead of writing the TXT at each timestep, we could store the data in memory and write everything at the last time step.

We could have a user argument to choose the behaviour.

@KulaginVladimir what do you think?

@RemDelaporteMathurin RemDelaporteMathurin added the enhancement New feature or request label Aug 2, 2024
@KulaginVladimir
Copy link
Collaborator

We could have a user argument to choose the behaviour.

Makes sense.

@KulaginVladimir
Copy link
Collaborator

KulaginVladimir commented Sep 8, 2024

We could have a user argument to choose the behaviour.

Do we need to write data at each timestep?

In the current implementation, TXTExport writes data -> reads old data and appends new -> writes data, and so on. The only benefit I see is that at some timestep we can run out of memory and get an error, but the already written data will remain (?). However, we nonetherless read the whole dataset at each call of TXTExport.write()

@RemDelaporteMathurin
Copy link
Collaborator Author

We are working with numpy arrays so it's unlikely that we will run out of memory.

Plus I don't see how not writing the data at each timestep would solve this? Maybe there's something I don't get here

@KulaginVladimir
Copy link
Collaborator

KulaginVladimir commented Sep 9, 2024

We are working with numpy arrays so it's unlikely that we will run out of memory.

I guess, it depends on the export parameters (number of export times and number of vertices).

Plus I don't see how not writing the data at each timestep would solve this? Maybe there's something I don't get here

It won't. I meant that there is no much difference. Maybe, just make TXTExport to write data at last export time?

@KulaginVladimir KulaginVladimir linked a pull request Sep 9, 2024 that will close this issue
14 tasks
@RemDelaporteMathurin RemDelaporteMathurin linked a pull request Oct 3, 2024 that will close this issue
14 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants