stdout output of shell command in python is truncated

I have a simple command where I’m processing a file and generating an output that is 25,000 lines.

sh some_command.sh > foo.txt | wc -l

Returns me a number showing that amount. Opening the file, I can see it also contains the same.

I am trying to replicate this in a python command:

First I tried:

result = subprocess.run(command, stdout=subprocess.PIPE)
print(result.stdout)

But the file was cut off around 22k lines.

I changed this to use:

process = subprocess.Popen(command, stdout=subprocess.PIPE)
  line = process.stdout.readline()
  print(line)

and also:

with open("/tmp/file.txt", 'w+') as out_file:
      result = subprocess.run(command, stdout=out_file)

and then read the contents using file.readline() or file.read()

and in each case the file was terminated. Is there any way I can read the output of this command in a python command?

You need to sign in to view this answers