A novel framework for multivariate time series representation learning based on the transformer encoder architecture.The framework includes an unsupervised pre-training scheme, which can offer substantial performance benefits over fully supervised learning on downstream tasks.