Deeplearning4j
Community ForumND4J JavadocDL4J Javadoc
EN 1.0.0-M2
EN 1.0.0-M2
  • Deeplearning4j Suite Overview
  • Release Notes
    • 1.0.0-M2
    • 1.0.0-M1.1
    • 1.0.0-M1
    • 1.0.0-beta7
    • 1.0.0-beta6
    • 1.0.0-beta5
    • 1.0.0-beta4
    • 1.0.0-beta3
    • 1.0.0-beta2
    • 1.0.0-beta
    • 1.0.0-alpha
    • 0.9.1
    • 0.9.0
    • 0.8.0
    • 0.7.2
    • 0.7.1
    • 0.7.0
    • 0.6.0
    • 0.5.0
    • 0.4.0
    • 1.00-M2.2
  • Multi-Project
    • Tutorials
      • Beginners
      • Quickstart
    • How To Guides
      • Import in to your favorite IDE
      • Contribute
        • Eclipse Contributors
      • Developer Docs
        • Github Actions/Build Infra
        • Javacpp
        • Release
        • Testing
      • Build From Source
      • Benchmark
      • Beginners
    • Reference
      • Examples Tour
    • Explanation
      • The core workflow
      • Configuration
        • Backends
          • Performance Issues
          • CPU
          • Cudnn
        • Memory
          • Workspaces
      • Build Tools
      • Snapshots
      • Maven
  • Deeplearning4j
    • Tutorials
      • Quick Start
      • Language Processing
        • Doc2Vec
        • Sentence Iterator
        • Tokenization
        • Vocabulary Cache
    • How To Guides
      • Custom Layers
      • Keras Import
        • Functional Models
        • Sequential Models
        • Custom Layers
        • Keras Import API Overview
          • Advanced Activations
          • Convolutional Layers
          • Core Layers
          • Embedding Layers
          • Local Layers
          • Noise Layers
          • Normalization Layers
          • Pooling Layers
          • Recurrent Layers
          • Wrapper Layers
        • Supported Features Overview
          • Activations
          • Constraints
          • Initializers
          • Losses
          • Optimizers
          • Regularizers
      • Tuning and Training
        • Visualization
        • Troubleshooting Training
        • Early Stopping
        • Evaluation
        • Transfer Learning
    • Reference
      • Model Zoo
        • Zoo Models
      • Activations
      • Auto Encoders
      • Computation Graph
      • Convolutional Layers
      • DataSet Iterators
      • Layers
      • Model Listeners
      • Saving and Loading Models
      • Multi Layer Network
      • Recurrent Layers
      • Updaters/Optimizers
      • Vertices
      • Word2vec/Glove/Doc2Vec
    • Explanation
  • datavec
    • Tutorials
      • Overview
    • How To Guides
    • Reference
      • Analysis
      • Conditions
      • Executors
      • Filters
      • Normalization
      • Operations
      • Transforms
      • Readers
      • Records
      • Reductions
      • Schemas
      • Serialization
      • Visualization
    • Explanation
  • Nd4j
    • Tutorials
      • Quickstart
    • How To Guides
      • Other Framework Interop
        • Tensorflow
        • TVM
        • Onnx
      • Matrix Manipulation
      • Element wise Operations
      • Basics
    • Reference
      • Op Descriptor Format
      • Tensor
      • Syntax
    • Explanation
  • Samediff
    • Tutorials
      • Quickstart
    • How To Guides
      • Importing Tensorflow
      • Adding Operations
        • codegen
    • Reference
      • Operation Namespaces
        • Base Operations
        • Bitwise
        • CNN
        • Image
        • LinAlg
        • Loss
        • Math
        • NN
        • Random
        • RNN
      • Variables
    • Explanation
      • Model Import Framework
  • Libnd4j
    • How To Guides
      • Building on Windows
      • Building for raspberry pi or Jetson Nano
      • Building on ios
      • How to Add Operations
      • How to Setup CLion
    • Reference
      • Understanding graph execution
      • Overview of working with libnd4j
      • Helpers Overview (CUDNN, OneDNN,Armcompute)
    • Explanation
  • Python4j
    • Tutorials
      • Quickstart
    • How To Guides
      • Write Python Script
    • Reference
      • Python Types
      • Python Path
      • Garbage Collection
      • Python Script Execution
    • Explanation
  • Spark
    • Tutorials
      • DL4J on Spark Quickstart
    • How To Guides
      • How To
      • Data How To
    • Reference
      • Parameter Server
      • Technical Reference
    • Explanation
      • Spark API Reference
  • codegen
Powered by GitBook
On this page
  • Introduction
  • Before you write your first script
  • Writing and executing your first script
  • Retrieving results

Was this helpful?

Export as PDF
  1. Python4j
  2. How To Guides

Write Python Script

How to write a python script for python4j

PreviousHow To GuidesNextReference

Last updated 3 years ago

Was this helpful?

Introduction

Writing a python script in python4j involves first understanding what variables you want to pass in and what variables you want to retrieve, very similar to writing any function in a programming language. In order to learn more about this, please see our

When writing a python script, a user should try to write the script to be as minimal as possible. Focus on the minimal set of inputs, outputs, and code you want to run within a python script. As this is an embedded interpreter, too many complexities arise when trying to run a full blown application. Some complexities include and

If you are using external libraries, then you need to understand how our works.

Before you write your first script

It is advised to test your python script in a real environment first. This would mean testing in a code editor and debugger for python like pycharm or visual studio code. This will also help you to determine what the script should look like for python4j. The following considerations should be thought about:

  1. inputs: the inputs to the script will be passed in from java and should not be declared as explicit variables in your script that's running embedded. These variable declarations will be dynamically created and inserted in to a real python script that gets executed by our execution framework.

  2. outputs: the outputs of the script will be passed from real python memory and are kept in memory within the scope of the try/with execution block.

  3. dependencies: the dependencies of your application should be bundled separately. The developers recommend a standalone miniconda installation for the target operating system. This version should match the version of cpython provided by python4j to avoid clashing. See [../reference/python-path] for more information on this topic.

Writing and executing your first script

Hello world is pretty straightforward. We'll do this write in line:

PythonExecutioner.exec("print('hello world')");

This will do as you would expect and print hello world to the console.

Next , we can add concateneate 2 strings. In this example, we pass hello world in as strings. Note that we pass in 2 variables of type string:

try(PythonGIL pythonGIL = PythonGIL.lock()) {
            List<PythonVariable> inputs = new ArrayList<>();
            inputs.add(new PythonVariable<>("x", PythonTypes.STR, "Hello "));
            inputs.add(new PythonVariable<>("y", PythonTypes.STR, "World"));
            String code = "print(x + y)";
            PythonExecutioner.exec(code, inputs, null);
        }

We could also pass in 2 ints and add them as well:

try(PythonGIL pythonGIL = PythonGIL.lock()) {
            List<PythonVariable> inputs = new ArrayList<>();
            inputs.add(new PythonVariable<>("x", PythonTypes.INT, 1));
            inputs.add(new PythonVariable<>("y", PythonTypes.INT, 2));
            String code = "print(x + y)";
            PythonExecutioner.exec(code, inputs, null);
        }

If we want to write an actual python script and have python4j load it, we need to read the script in to memory.

This can be achieved with the following:

    String code = FileUtils.readFileToString(new File("path/to/pythonfile.py"), StandardCharsets.UTF_8);

From there, we can pass the code to PythonExecutioner.exec(..) as follows:

try(PythonGIL pythonGIL = PythonGIL.lock()) {
            List<PythonVariable> inputs = new ArrayList<>();
            inputs.add(new PythonVariable<>("x", PythonTypes.INT, 1));
            inputs.add(new PythonVariable<>("y", PythonTypes.INT, 2));
             String code = FileUtils.readFileToString(new File("path/to/pythonfile.py"), StandardCharsets.UTF_8);
            PythonExecutioner.exec(code, inputs, null);
        }

Retrieving results

Up till now, we haven't actually retrieved results from the python script, just passed them in. Below is how to retrieve results:

    try(PythonGIL pythonGIL = PythonGIL.lock()) {
            String code = "a = 5\nb = '10'\nc = 20.0";
            List<PythonVariable> vars = PythonExecutioner.execAndReturnAllVariables(code);
        }

This will retrieve all results output from the executed python code. If you only want certain variables, then you can do the following:

   try(PythonGIL pythonGIL = PythonGIL.lock()) {
            List<PythonVariable> inputs = new ArrayList<>();
            inputs.add(new PythonVariable<>("x", PythonTypes.STR, "Hello "));
            inputs.add(new PythonVariable<>("y", PythonTypes.STR, "World"));
            PythonVariable out = new PythonVariable<>("z", PythonTypes.STR);
            String code = "z = x + y";
            PythonExecutioner.exec(code, inputs, Collections.singletonList(out));
        }

Afterwards, you can read the value from out post execution using:

String value = out.getValue();

The supported python types can be found More on types can be found

Note that out is a parameterized type. When retrieving the value, the java runtime will automatically try to cast whatever the output result is from python to the specified type. For more information on types, please see our

execution overview
garbage collection understanding
debugging script execution
custom python path support
here
here
types reference