HPCortex Documentation

Contents:

• Dependencies
• Installation
• CUDA + OpenMP (for NVIDIA V100 with compute capability 7.0):
• Pure OpenMP:
• OLCF Frontier:
• ALCF Aurora:
• Getting started
  • Application boilerplate and compilation
  • Building a simple model
  • Training the model
  • Inference

API:

• Accelerator
  • DO_PRAGMA_
  • DO_PRAGMA
  • thread_num
  • thread_max
  • set_threads
  • in_thread_parallel_region
  • thread_for
  • thread_for3d
  • thread_for2d
  • strong_inline
  • gen_lambda1
  • accelerator_for_gen
  • gen_lambda2
  • accelerator_for_2d_gen
  • gen_lambda3
  • accelerator_for_3d_gen
  • gen_lambda4
  • accelerator_for_4d_gen
  • gen_lambda5
  • accelerator_for_5d_gen
  • accelerator_for3dNB
  • accelerator_for3dNB_shm
  • accelerator_for_1_3_NB_shm
  • accelerator_for_1_3_shm
  • accelerator_for_1_3_NB
  • accelerator_for_1_3
  • accelerator_for_2_3_NB_shm
  • accelerator_for_2_3_shm
  • accelerator_for3d
  • accelerator_for2dNB
  • accelerator_for2d
  • accelerator_forNB
  • accelerator_for
  • accelerator_for3d_shm
  • accelerator_for2dNB_shm
  • accelerator_for2d_shm
  • accelerator_forNB_shm
  • accelerator_for_shm
  • autoView
  • doHost
  • doHost2
  • doHost3
  • doHost4
  • acceleratorInit()
  • acceleratorReport()
  • decomp
  • decomp< decompCoordPolicy, thrDims, blockDims, 1 >
  • decomp< decompCoordPolicy, thrDims, blockDims, 1 >::helper< w, true >
  • decomp< decompCoordPolicy, thrDims, blockDims, 1 >::helper< w, false >
  • loopOptions
  • viewDeallocator
• ActivationFuncs
  • ReLU
  • noActivation
  • GeLU
• Comms
  • communicators()
  • initializeComms()
  • UniqueID()
  • getMPIdataType()
  • commsReduce()
  • commsReduce()
  • commsBroadcast()
  • commsBroadcast()
  • commsBroadcast()
  • waitAll()
  • Communicators
  • PostCommActionCallback
  • PostCommActionCallbackUnlock
  • PostCommActionCallbackTensorInitialize
  • CommsRequest
• Components
• DDP
  • ddpAverage()
  • ddpAverage()
• DynamicModel
  • enwrap()
  • LayerWrapperInternalBase
  • LayerWrapperInternal
  • LayerWrapper
• Embeddings
  • embedPositionsSinusoidal()
  • embedPositionsSinusoidal()
• HPCortex
• Init
  • initialize()
• InstanceStorage
  • DDST
  • ISSTORAGE
  • INPUT_CON
  • enable_if_fwd_ref
  • enable_if_fwd_ref2
  • enable_if_fwd_ref3
  • StorageTag
  • LeafStore
  • LeafRef
  • deduceStorage
  • deduceStorage< T & >
  • deduceStorage< T && >
• Layers
• Linalg
  • thinMulMatMatTranspose_p()
  • thinMulMatMatTranspose()
  • mulMatTransposeThinMat()
  • computeThinMatOuterProd()
  • axpyMatThinMat()
  • batch3tensorContract()
  • matrixBatchTensorAxpy()
  • batchTensorContractToMatrix_p()
  • matrixBatchTensorContractRight()
  • matrixBatchTensorContractLeft()
• LossFunctions
  • CWRP
  • mse_cost()
  • cost_func_wrap()
  • cost_func_wrap()
  • CostFuncWrapper
  • MSEcostFunc
  • MSEcostFunc< Tensor< FloatType, Dim > >
• ManagedArray
  • ManagedArray
  • ManagedTypeArray
• MemoryManager
  • ViewMode
  • Locale
  • memPoolManagerReport()
  • MemoryManager
• Optimizers
  • train()
  • train()
  • batchData()
  • noScheduler
  • GradientDescentOptimizer
  • AdamParams
  • AdamState
  • AdamOptimizer
  • DecayScheduler
  • XYpair
  • XYpairDataLoader
• Padding
  • NoPadding
  • SamePaddingZero1D
• Performance
  • FLOPScounter
• Pipelining
• Random
  • GlobalRNGtype
  • globalRNG()
  • reseedGlobalRNG()
  • random()
  • uniformRandom()
  • uniformRandom()
  • glorotUniformRandom()
  • glorotUniformRandom()
  • drawWeightedRandomIndex()
  • drawWeightedRandomIndex()
  • default_seed
• RingBuffer
• Serialization
  • Endianness
  • toString()
  • endianness()
  • BitReverseTable256()
  • bitReverse()
  • BinaryWriter
  • BinaryReader
• Tensors
  • _1D_TENSOR_ONLY
  • _2D_TENSOR_ONLY
  • _3D_TENSOR_ONLY
  • _4D_TENSOR_ONLY
  • Vector
  • Matrix
  • pokeColumn()
  • pokeRow()
  • peekColumn()
  • peekColumns()
  • pokeColumns()
  • dimensionSlice()
  • dimensionSlice()
  • normalize()
  • unnormalize()
  • transpose()
  • operator<<()
  • operator<<()
  • operator*()
  • operator+=()
  • operator+()
  • operator-=()
  • operator-()
  • operator*=()
  • operator*()
  • operator*()
  • flatten()
  • flatten()
  • unflatten()
  • unflatten()
  • flatten2()
  • unflatten2()
  • flattenNsameDim()
  • unflattenNsameDim()
  • batchTensorConcatenate()
  • batchTensorSplit()
  • norm2()
  • transformBatchMatrix()
  • untransformBatchMatrix()
  • transformBatchVector()
  • untransformBatchVector()
  • Tensor
  • normalization
• Testing
  • near()
  • near()
  • near()
  • abs_near()
  • abs_near()
  • abs_near()
  • equal()
  • benchmark()
  • testCost()
  • testDeriv()
  • testComponentDeriv()
  • softMaxVector()
• Timing
  • TIME
  • now()
  • usSinceEpoch()
  • usCountSince()
  • since()
  • Timer
• Batch3tensorPairContractComponent
  • Batch3tensorPairContractComponent
• BatchedMatrixRowSoftMaxComponent
  • BatchedMatrixRowSoftMaxComponent
• BatchTensorConcatenateComponent
  • BatchTensorConcatenateComponent
• BatchTensorDimensionSliceComponent
  • BatchTensorDimensionSliceComponent
• BatchTensorDNNcomponent
  • BatchTensorDNNcomponent
• MatrixTensorContractComponent
  • MatrixTensorContractComponent
• MultiHeadAttentionComponent
  • MultiHeadAttentionComponent
• NormComponent
  • NormComponent
• ScaleComponent
  • ScaleComponent
• ScaledDotProductAttentionComponent
  • ScaledDotProductAttentionComponent
• ScaledDotProductAttentionHeadComponent
  • ScaledDotProductAttentionHeadComponent
• SoftMaxComponent
  • SoftMaxComponent
• BatchTensorDNNlayer
  • LAYER_TYPE
  • batch_tensor_dnn_layer()
  • batch_tensor_dnn_layer()
  • batch_tensor_dnn_layer()
  • batch_tensor_unbiased_dnn_layer()
  • dnn_layer()
  • dnn_layer()
  • dnn_layer()
  • dnn_layer()
  • BatchTensorDNNlayer
• ConvolutionLayer1D
  • conv1d_layer()
  • conv1d_layer()
  • ConvolutionLayer1D
• EmbedPositionsSinusoidalLayer
  • embed_positions_sinusoidal_layer()
  • EmbedPositionsSinusoidalLayer
• FlattenLayer
  • flatten_layer()
  • FlattenLayer
• InputLayer
  • input_layer()
  • InputLayer
• LayerCommon
  • ISLEAF
  • FLOATTYPE
  • INPUTTYPE
  • LAYEROUTPUTTYPE
  • LAYERTYPEOUTPUTTYPE
  • CONFIGTYPE
  • void_type
  • is_leaf_v
  • LeafTag
  • is_leaf
  • is_leaf< T, void_type< typename std::decay_t< T >::tag > >
• MatrixTensorContractLayer
  • matrix_tensor_contract_layer()
  • MatrixTensorContractLayer
• MultiHeadCrossAttentionLayer
  • TEMPL
  • multihead_cross_attention_layer()
  • multihead_cross_attention_layer()
  • multihead_cross_attention_layer()
  • multihead_cross_attention_layer()
  • multihead_cross_attention_layer()
  • multihead_cross_attention_layer()
  • MultiHeadCrossAttentionLayer
• MultiHeadSelfAttentionLayer
  • multihead_self_attention_layer()
  • multihead_self_attention_layer()
  • multihead_self_attention_layer()
  • multihead_self_attention_layer()
  • multihead_self_attention_layer()
  • multihead_self_attention_layer()
  • MultiHeadSelfAttentionLayer
• NormLayer
  • norm_layer()
  • norm_layer()
  • NormLayer
• PairJoinLayer
  • pair_join_layer()
  • PairJoinLayer
• PairSplitLayer
  • pair_split_layer()
  • PairSplitLayerLeader
  • PairSplitLayer1
  • PairSplitLayer2
• ReplicateLayer
  • replicate_layer()
  • ReplicateLayerLeader
  • ReplicateLayer
• ScaledDotProductSelfAttentionLayer
  • scaled_dotproduct_self_attention_layer()
  • scaled_dotproduct_self_attention_layer()
  • ScaledDotProductSelfAttentionLayer
• SkipConnection
  • skip_connection()
  • SkipConnection
• SoftMaxLayer
  • softmax_layer()
  • SoftMaxLayer
• SumJoinLayer
  • sum_join_layer()
  • SumJoinLayer
• TransformerEncoderDecoderBlock
  • transformer_decoder_block()
  • transformer_encoder_block()
  • transformer_cross_decoder_block()
• UnflattenLayer
  • unflatten_layer()
  • UnflattenLayer