Aging has a massive effect on the features and appearance of the human face. Even though various traits are utilized to estimate human age, this article focuses on age classification using prominent texture and edge-based feature vectors. Most of the local-based methods derive features on a circular window. The applications related to facial skin require the computation of elliptical or anisotropic features since the lips, eyes, and other prominent facial skin features primarily represent elliptical shapes. Further, the locallybased approaches are prevalent in estimating human age; however, most of these methods are intensitybased and sensitive to noise illumination changes, thus may not provide better results. To address this, the local directional patterns (LDP) are proposed, which derives features based on the top edge responses in all eight directions in the form of binary patterns of a 3x3 window. The disadvantage of LDP is finding the threshold for top edge responses. This paper derived an automatic process for deriving thresholds and explored the derivation of the ternary pattern instead of binary patterns. To reduce the complexity, the 3x3 window is divided into cross diagonal -local direction Ternary matrix (CD-LDTM) on both isometric (ICDLDTM) and anisometric (ACD-LDTM) local structures. The facial features derived by the proposed ICDLDTM and ACD-LDTM descriptors are fed to machine learning classifiers for age classification purposes. The experimental results demonstrate that the proposed strategy is effective.