Contents Menu Expand Light mode Dark mode Auto light/dark mode Auto light/dark, in light mode Auto light/dark, in dark mode
Flyte
Logo
Flyte
  • Introduction
  • Quickstart guide
  • Getting started with workflow development
    • Installing development tools
    • Creating a Flyte project
    • Flyte project components
    • Running a workflow locally
  • Flyte fundamentals
    • Tasks, workflows and launch plans
    • Registering workflows
    • Running and scheduling workflows
    • Visualizing task input and output
    • Optimizing tasks
    • Extending Flyte
  • Flyte agents
    • Testing agents in a local Python environment
    • Enabling agents in your Flyte deployment
    • Developing agents
    • Testing agents in a local development cluster
    • Deploying agents to the Flyte sandbox
    • Implementing the agent metadata service
    • How Secret Works in Agent
  • Core use cases
    • Data engineering
    • Machine learning
    • Analytics

Examples

  • User Guide
    • Environment setup
    • Basics
      • Hello, World!
      • Tasks
      • Workflows
      • Launch plans
      • Imperative workflows
      • Documenting workflows
      • Shell tasks
      • Named outputs
    • Data Types and IO
      • FlyteFile
      • FlyteDirectory
      • StructuredDataset
      • Dataclass
      • Accessing attributes
      • PyTorch type
      • Enum type
      • Pickle type
    • Advanced composition
      • Conditionals
      • Chaining Flyte entities
      • Subworkflows
      • Dynamic workflows
      • Map tasks
      • Nested parallelization
      • Eager workflows
      • Decorating tasks
      • Decorating workflows
      • Intratask checkpoints
      • Waiting for external inputs
    • Customizing dependencies
      • ImageSpec
      • Raw containers
      • Multiple images in a workflow
    • Development lifecycle
      • Private images
      • Caching
      • Cache serializing
      • Decks
      • Failure node
      • Creating a new project
      • Running tasks
      • Running workflows
      • Running launch plans
      • Inspecting executions
      • Debugging executions
      • Migrating from Airflow to Flyte
      • For current Flyte users
    • Testing
      • Mocking tasks
    • Productionize
      • Customizing task resources
      • Reference tasks
      • Reference launch plans
      • Notifications
      • Schedules
      • Configuring logging links in the UI
      • Configuring access to GPUs
      • Spot instances
      • Secrets
      • Workflow labels and annotations
    • Extending Flyte
      • Custom types
      • Prebuilt container task plugins
      • User container task plugins
      • Backend plugins
      • Container interface
  • Deprecated integrations
    • BigQuery plugin
      • BigQuery example query
    • Databricks plugin
      • Running Spark on Databricks
    • Snowflake plugin
      • Querying data in Snowflake

Cluster Deployment

  • Getting Started
  • Deployment Paths
    • Sandbox Deployment
    • Single Cluster Simple Cloud Deployment
    • Single Cluster Production-grade Cloud Deployment
    • Multiple Kubernetes Cluster Deployment
  • Plugin Setup
    • Configure Kubernetes Plugins
    • Configure AWS Plugins
      • AWS Batch
      • Athena Plugin
      • Sagemaker Plugin Setup
    • Configure GCP Plugins
      • Google BigQuery Plugin
    • Configure Web APIs
      • Snowflake Plugin
      • Databricks Plugin
  • Agent Setup
    • Airflow agent
    • ChatGPT agent
    • Databricks agent
    • Google BigQuery agent
    • MMCloud Agent
    • SageMaker Inference Agent
    • Sensor agent
    • Snowflake agent
    • OpenAI Batch Agent
  • Configuration
    • Authenticating in Flyte
    • Migrating Your Authentication Config
    • Understanding Authentication
    • Configuring task pods with K8s PodTemplates
    • Customizing project, domain, and workflow resources with flytectl
    • Monitoring
    • Notifications
    • Optimizing Performance
    • Cloud Events
    • Flyte ResourceManager
  • Configuration Reference
    • Flyte Datacatalog Configuration
    • Flyte Admin Configuration
    • Flyte Propeller Configuration
    • Flyte Scheduler Configuration
  • Security Overview
  • Flyte API Playground: Swagger

API Reference

  • flytekit
    • Overview
      • Model Files
      • Authoring Structure
      • Control Plane: FlyteRemote
      • Execution Time Support
      • Command Line Interfaces and Clients
    • Core Flytekit
      • flytekit.task
      • flytekit.workflow
      • flytekit.kwtypes
      • flytekit.current_context
      • flytekit.ExecutionParameters
      • flytekit.FlyteContext
      • flytekit.map_task
      • flytekit.core.workflow.ImperativeWorkflow
      • flytekit.core.node_creation.create_node
      • flytekit.core.promise.NodeOutput
      • flytekit.FlyteContextManager
      • flytekit.conditional
      • flytekit.TaskMetadata
      • flytekit.Resources
      • flytekit.WorkflowFailurePolicy
      • flytekit.PodTemplate
      • flytekit.dynamic
      • flytekit.approve
      • flytekit.sleep
      • flytekit.wait_for_input
      • flytekit.CronSchedule
      • flytekit.FixedRate
      • flytekit.Email
      • flytekit.PagerDuty
      • flytekit.Slack
      • flytekit.get_reference_entity
      • flytekit.LaunchPlanReference
      • flytekit.TaskReference
      • flytekit.WorkflowReference
      • flytekit.reference_task
      • flytekit.reference_workflow
      • flytekit.reference_launch_plan
      • flytekit.SQLTask
      • flytekit.ContainerTask
      • flytekit.PythonFunctionTask
      • flytekit.PythonInstanceTask
      • flytekit.LaunchPlan
      • flytekit.Secret
      • flytekit.SecurityContext
      • flytekit.AuthRole
      • flytekit.Labels
      • flytekit.Annotations
      • flytekit.WorkflowExecutionPhase
      • flytekit.Blob
      • flytekit.BlobMetadata
      • flytekit.Literal
      • flytekit.Scalar
      • flytekit.LiteralType
      • flytekit.BlobType
      • flytekit.HashMethod
      • flytekit.Artifact
      • flytekit.Description
      • flytekit.Documentation
      • flytekit.SourceCode
    • Configuration
      • flytekit.configuration.Config
      • flytekit.configuration.Image
      • flytekit.configuration.ImageConfig
      • flytekit.configuration.SerializationSettings
      • flytekit.configuration.FastSerializationSettings
      • flytekit.configuration.PlatformConfig
      • flytekit.configuration.StatsConfig
      • flytekit.configuration.SecretsConfig
      • flytekit.configuration.S3Config
      • flytekit.configuration.GCSConfig
      • flytekit.configuration.DataConfig
    • ImageSpec
      • flytekit.image_spec.ImageSpec
    • Remote Access
      • flytekit.remote.remote.FlyteRemote
      • flytekit.remote.remote.Options
      • flytekit.remote.entities.FlyteTask
      • flytekit.remote.entities.FlyteWorkflow
      • flytekit.remote.entities.FlyteLaunchPlan
      • flytekit.remote.entities.FlyteNode
      • flytekit.remote.entities.FlyteTaskNode
      • flytekit.remote.entities.FlyteWorkflowNode
      • flytekit.remote.executions.FlyteWorkflowExecution
      • flytekit.remote.executions.FlyteTaskExecution
      • flytekit.remote.executions.FlyteNodeExecution
    • Clients
      • flytekit.clients.friendly.SynchronousFlyteClient
      • flytekit.clients.raw.RawSynchronousFlyteClient
    • Unit Testing
      • flytekit.testing.patch
      • flytekit.testing.task_mock
    • Specifying Accelerators
      • flytekit.extras.accelerators.BaseAccelerator
      • flytekit.extras.accelerators.GPUAccelerator
      • flytekit.extras.accelerators.MultiInstanceGPUAccelerator
      • flytekit.extras.accelerators.A10G
      • flytekit.extras.accelerators.L4
      • flytekit.extras.accelerators.K80
      • flytekit.extras.accelerators.M60
      • flytekit.extras.accelerators.P4
      • flytekit.extras.accelerators.P100
      • flytekit.extras.accelerators.T4
      • flytekit.extras.accelerators.V100
      • flytekit.extras.accelerators.A100
      • flytekit.extras.accelerators.A100_80GB
    • Extending Flytekit
      • flytekit.extend.get_serializable
      • flytekit.extend.context_manager
      • flytekit.extend.IgnoreOutputs
      • flytekit.extend.ExecutionState
      • flytekit.extend.Image
      • flytekit.extend.ImageConfig
      • flytekit.extend.Interface
      • flytekit.extend.Promise
      • flytekit.extend.TaskPlugins
      • flytekit.extend.DictTransformer
      • flytekit.extend.T
      • flytekit.extend.TypeEngine
      • flytekit.extend.TypeTransformer
      • flytekit.extend.PythonCustomizedContainerTask
      • flytekit.extend.ExecutableTemplateShimTask
      • flytekit.extend.ShimTaskExecutor
    • Flyte Deck
      • flytekit.deck.Deck
      • flytekit.deck.TopFrameRenderer
      • flytekit.deck.MarkdownRenderer
      • flytekit.deck.SourceCodeRenderer
    • Plugin API reference
      • AWS Athena
        • flytekitplugins.athena.AthenaConfig
        • flytekitplugins.athena.AthenaTask
      • AWS Batch
        • flytekitplugins.awsbatch.AWSBatchConfig
      • Google Bigquery
        • flytekitplugins.bigquery.BigQueryConfig
        • flytekitplugins.bigquery.BigQueryTask
        • flytekitplugins.bigquery.BigQueryAgent
      • Dask
        • flytekitplugins.dask.Dask
        • flytekitplugins.dask.Scheduler
        • flytekitplugins.dask.WorkerGroup
      • Deck standard
        • flytekitplugins.deck.BoxRenderer
        • flytekitplugins.deck.FrameProfilingRenderer
        • flytekitplugins.deck.GanttChartRenderer
        • flytekitplugins.deck.ImageRenderer
        • flytekitplugins.deck.MarkdownRenderer
        • flytekitplugins.deck.SourceCodeRenderer
        • flytekitplugins.deck.TableRenderer
      • Dolt standard
        • flytekitplugins.dolt.DoltConfig
        • flytekitplugins.dolt.DoltTable
        • flytekitplugins.dolt.DoltTableNameTransformer
      • Great expectations
        • flytekitplugins.great_expectations.BatchRequestConfig
        • flytekitplugins.great_expectations.GreatExpectationsFlyteConfig
        • flytekitplugins.great_expectations.GreatExpectationsTask
        • flytekitplugins.great_expectations.GreatExpectationsType
      • Hive
        • flytekitplugins.hive.HiveConfig
        • flytekitplugins.hive.HiveSelectTask
        • flytekitplugins.hive.HiveTask
      • Pod
      • KF MPI
        • flytekitplugins.kfmpi.MPIJob
      • KF Pytorch
        • flytekitplugins.kfpytorch.PyTorch
        • flytekitplugins.kfpytorch.Elastic
      • KF Tensorflow
        • flytekitplugins.kftensorflow.TfJob
      • Modin
        • flytekitplugins.modin.ModinPandasDataFrameTransformer
      • Pandera
        • flytekitplugins.pandera.PanderaPandasTransformer
        • flytekitplugins.pandera.PandasReportRenderer
        • flytekitplugins.pandera.ValidationConfig
      • Papermill
        • flytekitplugins.papermill.NotebookTask
        • flytekitplugins.papermill.record_outputs
      • PERIAN
      • Snowflake
        • flytekitplugins.snowflake.SnowflakeConfig
        • flytekitplugins.snowflake.SnowflakeTask
        • flytekitplugins.snowflake.SnowflakeAgent
      • Spark
        • flytekitplugins.spark.new_spark_session
        • flytekitplugins.spark.ParquetToSparkDecodingHandler
        • flytekitplugins.spark.Spark
        • flytekitplugins.spark.SparkDataFrameSchemaReader
        • flytekitplugins.spark.SparkDataFrameSchemaWriter
        • flytekitplugins.spark.SparkDataFrameTransformer
      • SQLAlchemy
        • flytekitplugins.sqlalchemy.SQLAlchemyConfig
        • flytekitplugins.sqlalchemy.SQLAlchemyDefaultImages
        • flytekitplugins.sqlalchemy.SQLAlchemyTask
      • ONNX PyTorch
        • flytekitplugins.onnxpytorch.PyTorch2ONNXConfig
        • flytekitplugins.onnxpytorch.PyTorch2ONNX
      • ONNX TensorFlow
        • flytekitplugins.onnxtensorflow.TensorFlow2ONNXConfig
        • flytekitplugins.onnxtensorflow.TensorFlow2ONNX
      • ONNX ScikitLearn
        • flytekitplugins.onnxscikitlearn.ScikitLearn2ONNXConfig
        • flytekitplugins.onnxscikitlearn.ScikitLearn2ONNX
      • Ray
        • flytekitplugins.ray.HeadNodeConfig
        • flytekitplugins.ray.RayJobConfig
        • flytekitplugins.ray.WorkerNodeConfig
      • DBT
        • flytekitplugins.dbt.DBTRun
        • flytekitplugins.dbt.DBTTest
        • flytekitplugins.dbt.BaseDBTInput
        • flytekitplugins.dbt.BaseDBTOutput
        • flytekitplugins.dbt.DBTRunInput
        • flytekitplugins.dbt.DBTRunOutput
        • flytekitplugins.dbt.DBTTestInput
        • flytekitplugins.dbt.DBTTestOutput
      • Vaex
        • flytekitplugins.vaex.VaexDataFrameToParquetEncodingHandler
        • flytekitplugins.vaex.ParquetToVaexDataFrameDecodingHandler
      • MLflow
        • flytekitplugins.mlflow.mlflow_autolog
      • DuckDB
        • flytekitplugins.duckdb.DuckDBQuery
      • SageMaker Inference
        • flytekitplugins.awssagemaker_inference.BotoAgent
        • flytekitplugins.awssagemaker_inference.BotoTask
        • flytekitplugins.awssagemaker_inference.SageMakerModelTask
        • flytekitplugins.awssagemaker_inference.SageMakerEndpointConfigTask
        • flytekitplugins.awssagemaker_inference.SageMakerEndpointAgent
        • flytekitplugins.awssagemaker_inference.SageMakerEndpointTask
        • flytekitplugins.awssagemaker_inference.SageMakerDeleteEndpointConfigTask
        • flytekitplugins.awssagemaker_inference.SageMakerDeleteEndpointTask
        • flytekitplugins.awssagemaker_inference.SageMakerDeleteModelTask
        • flytekitplugins.awssagemaker_inference.SageMakerInvokeEndpointTask
        • flytekitplugins.awssagemaker_inference.create_sagemaker_deployment
        • flytekitplugins.awssagemaker_inference.delete_sagemaker_deployment
      • OpenAI
        • flytekitplugins.openai.BatchEndpointAgent
        • flytekitplugins.openai.BatchEndpointTask
        • flytekitplugins.openai.BatchResult
        • flytekitplugins.openai.DownloadJSONFilesTask
        • flytekitplugins.openai.UploadJSONLFileTask
        • flytekitplugins.openai.OpenAIFileConfig
        • flytekitplugins.openai.create_batch
        • flytekitplugins.openai.ChatGPTAgent
        • flytekitplugins.openai.ChatGPTTask
      • Inference
        • flytekitplugins.inference.NIM
        • flytekitplugins.inference.NIMSecrets
        • flytekitplugins.inference.Model
        • flytekitplugins.inference.Ollama
      • Neptune
      • Weights & Biases
    • Custom Tasks
      • flytekit.core.base_task.kwtypes
      • flytekit.core.base_task.PythonTask
      • flytekit.core.base_task.Task
      • flytekit.core.base_task.TaskResolverMixin
      • flytekit.core.base_task.IgnoreOutputs
      • flytekit.core.python_function_task.PythonFunctionTask
      • flytekit.core.python_function_task.PythonInstanceTask
      • Shell Task
        • flytekit.extras.tasks.shell.ShellTask
        • flytekit.extras.tasks.shell.OutputLocation
      • SQLite3 Task
        • flytekit.extras.sqlite3.task.SQLite3Task
        • flytekit.extras.sqlite3.task.SQLite3Config
    • Custom Types
      • Flytekit StructuredDataset
        • flytekit.types.structured.StructuredDataset
        • flytekit.types.structured.StructuredDatasetEncoder
        • flytekit.types.structured.StructuredDatasetDecoder
      • Flytekit File Type
        • flytekit.types.file.FlyteFile
        • flytekit.types.file.HDF5EncodedFile
        • flytekit.types.file.HTMLPage
        • flytekit.types.file.JoblibSerializedFile
        • flytekit.types.file.JPEGImageFile
        • flytekit.types.file.PDFFile
        • flytekit.types.file.PNGImageFile
        • flytekit.types.file.PythonPickledFile
        • flytekit.types.file.PythonNotebook
        • flytekit.types.file.SVGImageFile
      • Flytekit Directory Type
        • flytekit.types.directory.FlyteDirectory
        • flytekit.types.directory.TensorboardLogs
        • flytekit.types.directory.TFRecordsDirectory
      • Flytekit Iterator Type
        • flytekit.types.iterator.FlyteIterator
        • flytekit.types.iterator.JSON
      • PyTorch Type
        • flytekit.extras.pytorch.PyTorchCheckpoint
        • flytekit.extras.pytorch.PyTorchCheckpointTransformer
        • flytekit.extras.pytorch.PyTorchModuleTransformer
        • flytekit.extras.pytorch.PyTorchTensorTransformer
      • TensorFlow Type
        • flytekit.extras.tensorflow.TensorFlowRecordFileTransformer
        • flytekit.extras.tensorflow.TensorFlowRecordsDirTransformer
      • Sklearn Type
        • flytekit.extras.sklearn.SklearnEstimatorTransformer
    • Experimental Features
      • flytekit.experimental.map_task
      • flytekit.experimental.eager
      • flytekit.experimental.EagerException
    • Pyflyte CLI
    • Flytekit Contribution Guide
  • flytectl
    • Overview
    • CLI Entrypoint
    • Verbs
      • flytectl create
      • flytectl completion
      • flytectl get
      • flytectl update
      • flytectl delete
      • flytectl register
      • flytectl config
      • flytectl compile
      • flytectl sandbox
      • flytectl demo
      • flytectl version
      • flytectl upgrade
    • Nouns
      • Project
        • flytectl create project
        • flytectl get project
        • flytectl update project
      • Execution
        • flytectl create execution
        • flytectl get execution
        • flytectl update execution
        • flytectl delete execution
      • Workflow
        • flytectl get workflow
        • flytectl update workflow-meta
      • Task
        • flytectl get task
        • flytectl update task-meta
      • Task resource attribute
        • flytectl get task-resource-attribute
        • flytectl update task-resource-attribute
        • flytectl delete task-resource-attribute
      • Cluster resource attribute
        • flytectl get cluster-resource-attribute
        • flytectl delete cluster-resource-attribute
        • flytectl update cluster-resource-attribute
      • Execution cluster label
        • flytectl get execution-cluster-label
        • flytectl update execution-cluster-label
        • flytectl delete execution-cluster-label
      • Execution queue attribute
        • flytectl get execution-queue-attribute
        • flytectl delete execution-queue-attribute
        • flytectl update execution-queue-attribute
      • Plugin override
        • flytectl get plugin-override
        • flytectl update plugin-override
        • flytectl delete plugin-override
      • Launchplan
        • flytectl get launchplan
        • flytectl update launchplan
        • flytectl update launchplan-meta
      • Workflow execution config
        • flytectl get workflow-execution-config
        • flytectl update workflow-execution-config
        • flytectl delete workflow-execution-config
      • Examples
        • flytectl register examples
      • Files
        • flytectl register files
      • Config
        • flytectl config validate
        • flytectl config init
        • flytectl config docs
        • flytectl config discover
      • Sandbox
        • flytectl sandbox start
        • flytectl sandbox status
        • flytectl sandbox teardown
        • flytectl sandbox exec
      • Demo
        • flytectl demo start
        • flytectl demo status
        • flytectl demo teardown
        • flytectl demo exec
        • flytectl demo reload
    • Contributing Guide
  • flyteidl
    • Flyte Core Language Specification
    • FlyteAdmin Service
    • FlyteAdmin REST and gRPC Interface
    • Data Catalog Service
    • Internal and External Eventing Interface
    • Flyte Task Plugin Specification
    • Contributing Guide

Ecosystem

  • flytekit-java
  • unionml
  • pterodactyl
  • latch sdk

Community

  • Community Resources
  • Contributing to Flyte
  • Roadmap
  • Frequently Asked Questions
  • Troubleshooting Guide

Glossary

  • Main Concepts
    • Tasks
    • Workflows
    • Nodes
    • Launch plans
    • Schedules
    • Registration
    • Executions
    • Understanding the State Transition in a Workflow
    • Timeline of a workflow execution
    • Understand How Flyte Handles Data
    • How to Use Flyte UI
    • What is Data Catalog?
    • Versions
    • Understand the Lifecycle of a Flyte Workflow
  • Control Plane
    • Projects
    • Domains
    • FlyteAdmin
    • FlyteConsole
    • Dynamic Job Spec
  • Component Architecture
    • FlytePropeller Architecture
    • Flyte Native Scheduler Architecture
Flyte
Back to top

TensorFlow TypeΒΆ

TensorFlowRecordFileTransformer()

TypeTransformer that supports serialising and deserialising to and from TFRecord file.

TensorFlowRecordsDirTransformer()

TypeTransformer that supports serialising and deserialising to and from TFRecord directory.
Next
flytekit.extras.tensorflow.TensorFlowRecordFileTransformer
Previous
flytekit.extras.pytorch.PyTorchTensorTransformer
Copyright © 2022, Flyte Authors
flytekit: dev, 5a8941b
flytesnacks: dev, 8ef4220
flytectl: ,