Staff

Abstract:
Structural vibrations have adverse effects and can lead to catastrophic failures. Among various methods for mitigation of vibrations, the semi-active control approaches have the advantage of not requiring a large external power supply. In this paper, we propose and test a sliding mode control method for the semi-active mitigation of vibrations in frame structures. The control forces are generated in a purely dissipative manner by means of on/off type actuators that take the form of controllable structural nodes. These nodes are essentially lockable hinges, modeled as viscous dampers, which are capable of the on/off control of the transmission of bending moments between the adjacent beams. The control aim is formulated in terms of the displacement of a selected degree of freedom. A numerically effective model of such a node is developed, and the proposed control method is verified in a numerical experiment of a four-story shear structure subjected to repeated random seismic excitations. In terms of the root-mean-square displacement, the control reduced the response by 48.4-78.4% on average, depending on the number and placement of the applied actuators. The peak mean amplitude at the first mode of natural vibrations was reduced by as much as 70.6-96.5%. Such efficiency levels confirm that the proposed control method can effectively mitigate vibrations in frame structures.

Keywords:
semi-active control,sliding mode control,structural control,controllable nodes,on/off nodes,damping of vibrations

Abstract:
This study applies the machine learning technique of multi-agent reinforcement learning for semi-active structural control. The considered structure is a high-rise shear-type building subjected to seismic excitation, where actuators function as viscous dampers with a controllable level of damping. The problem formulation is inherently nonlinear due to the bilinear nature of the control concept. The analytical derivation of optimal semi-active control solutions is seldom feasible, leading to many practical control algorithms being suboptimal and/or heuristic in their formulation. In the framework proposed here, the control algorithm is developed through interaction with the controlled system by applying actions, observing results, and optimizing effects. For this purpose, a multi-agent reinforcement learning Q-network architecture is employed. Verification is conducted through a numerical experiment, utilizing a finite element model of a structure equipped with a tuned mass damper and a controllable viscous damper. The results demonstrate that the proposed method outperforms a conventional, optimally tuned TMD. This study also includes a comparison of the obtained results with those achieved using the typical single-agent approach. The key contribution is to demonstrate a significant improvement in control performance through the application of a multi-agent policy.

Keywords:
Structural control, Reinforcement Learning, Multi-agent, Damping, Vibration

Abstract:
This contribution applies the machine learning technique of reinforcement learning for simultaneous damage detection and control of structures. The proposed system consists of two components. The control component is responsible for semi-active mitigation of vibrations. The control law is determined experimentally in a trial-and-error interaction with a simulated environment. The process is data-driven: the control agent iteratively improves its control law based on the observed results of past control actions. The robustness relies on the accuracy of the structural model used for training. The control efficiency can decrease if the physical structure is damaged and diverges from the model, that is, when effective control may be most required. Thus, the second component of the proposed system monitors the structure to detect damages and inform the control component. The approach is tested in a numerical experiment of a shear-building under random seismic-type excitation. A semi-active tuned mass damper (TMD) is used as an actuator, and a classical TMD serves as a reference.

Keywords:
semi-active control, structural control, structural monitoring, reinforcement learning, machine learning

Abstract:
This contribution presents a semi-active control technique intended for mitigation of structural vibrations. The control law is derived in a repeated trial-and-error interaction between the control agent and a simulated environment. The experience-based training approach is used which is the defining feature of the machine learning techniques of reinforcement learning (RL), implemented here using the framework provided by Deep Q Learning (DQN). The involved artificial neural network not only determines the control action, but additionally identifies structural damages, which is a nontrivial task due to the nonlinearity of the control. This requires a specific multi-head architecture, which allows the network to be damage-aware, and a specific training procedure, where the memory pool preserved for the RL stage of experience replay is populated with not only the observations, control actions, and rewards, but also with the momentary status of structural damage. Such an approach can be used to explicitly promote the damage-awareness of the control agent. The proposed technique is tested and verified in a numerical example of a shear-type building model subjected to a random seismic-type excitation. A tuned mass damper (TMD) with a controllable level of viscous damping is used to implement the semi-active actuation, and the optimally tuned classical TMD provides the reference response.

Keywords:
semi-active control, tuned mass damper (TMD), reinforcement learning, damage identification

Abstract:
This paper considers structural control by reinforcement learning. The aim is to mitigate vibrations of a shear building subjected to an earthquake-like excitation and fitted with a semi-active tuned mass damper (TMD). The control force is coupled with the structural response, making the problem intrinsically nonlinear and challenging to solve using classical methods. Structural control by reinforcement learning has not been extensively explored yet. Here, Deep-Q-Learning is used, which appriximates the Q-function with a neural network and optimizes initially random control sequences through interaction with the controlled system. For safety reasons, training must be performed using an inevitably inexact numerical model instead of the real structure. It is thus crucial to assess the robustness of the control with respect to measurement noise and model errors. It is verified to significantly outperform an optimally tuned conventional TMD, and the key outcome is the high robustness to measurement noise and model error.

Keywords:
structural control, semi-active control, reinforcement learning, tuned mass damper (TMD)

Abstract:
This contribution presents an approach to structural control based on reinforcement learning. Reinforcement learning, a rapidly developing branch of machine learning, is based on the paradigm of learning through interaction with the environment. Here, it is applied in the context of semi-active structural control, where the considered actuators take the form of viscous dampers with a controllable level of damping. The control forces are thus coupled with the structural response, and the formulation is intrinsically nonlinear. The related optimum control problems are usually more difficult than in the case of active structural control systems, which generate and apply arbitrary external control forces. Analytical derivation of the optimum semi-active control is thus rarely possible, so that many control algorithms applied in practice are suboptimal and/or heuristic in nature. Here, an effective control strategy is developed by means of the Q-learning approach. The control algorithm is determined in interaction with the controlled system, that is, by applying initially random control sequences in order to observe, process, and optimize their effects. Such an approach seems to be new and relatively unexplored in the field of structural control. Verification is performed in a numerical experiment, where the Q-learning procedures interact with an independently simulated finite element model of a structure equipped with a tuned mass damper (TMD) and a controllable viscous damper. The results attest to a performance significantly better than that of an optimally tuned conventional TMD.

Keywords:
Reinforcement Learning, Semi-active control, Structural control, Damping, Vibration

Abstract:
This contribution presents a sliding-mode control approach for the mitigation of vibrations in frame-like structures. The control is implemented in a semi-active manner, that is, without significant external control forces and substantial power consumption, which are typical for active control approaches. Here, the control is achieved through dynamic, lowcost modification of properties at selected structural nodes. The employed actuators have the untypical form of two-state hinges, which can switch between two extreme states: no transfer of bending moments (effectively a hinge) and full transfer of bending moments (a locked hinge or a typical frame node). Consequently, the control forces are dissipative and coupled to the response. Previous research in this area focused on purely energetic considerations, aiming for global damping of vibrations. In contrast, this paper formulates the control objective in terms of local displacements of a selected degree of freedom, which can be interpreted as the task of isolating it from external excitations. This formulation is employed to define the target sliding hyperplane. The state of the actuators is chosen such that the effective control forces push the structural state toward the target hyperplane. The approach is verified in a numerical example of a six-story shear-type structure subjected to random seismic excitation.

Keywords:
Structural control, Semi-active control, Sliding mode control, On/off nodes

Abstract:
This contribution discusses an approach for semi-active control of structures based on the machine learning technique of reinforcement learning (RL). The RL family of techniques was recently demonstrated to lead to a super-human performance in many demanding control-like real-world problems, but the applications in the field of structural control are still sparse. Here, a seismically excited shear-type structure is considered and equipped with a tuned mass damper (TMD), where the optimally tuned passive TMD serves as the baseline reference case. Its effectiveness is compared to the considered control system, which, instead of keeping the TMD’s dashpot damping constant, switches it between two values that represent a low- and high-damping state. The control function is modeled using an artificial neural network and learned in a simulated series of trial-and-error experiments. The input to the network are the sensor readings, the output is the control signal, and the learning aims to maximize the control effectiveness defined in typical terms of total mechanical energy and interstory displacement. Since switching on the viscous damping may lead to undesirable impact-like excitations, an additional penalty term is introduced to mitigate the impact effects based on acceleration readings. The investigated RL-derived control is demonstrated to surpass the baseline and be relatively robust to measurement errors, but this robustness depends on the sensor redundancy.

The authors gratefully acknowledge the support of the National Science Centre, Poland, granted under the grant agreement 2020/39/B/ST8/02615. For the purpose of Open Access, the authors have applied a CC-BY public copyright licence to any Author Accepted Manuscript (AAM) version arising from this submission.

Abstract:
Reinforcement learning (RL) is a class of machine learning methods aimed at solving control-like tasks. Control is learned and applied by an agent that learns to select the best action through a series of interactions with the controlled system.

A potential application area of reinforcement learning is semi-active structural control. In such tasks, the optimal control signals are often unknown and cannot thus be used for training, as required by typical supervised learning techniques. The RL approach is more suitable, as it allows the control agent to progressively improve its decision-making strategy in a trial-and-error manner based on experiencing and received feedback (reward). This interaction-based approach also goes beyond typical unsupervised techniques, as it allows control signals to be actively tested rather than merely exploring pre-prepared datasets. Despite these advantages and remarkable achievements in other fields, reinforcement learning methods are still rarely applied in structural control, especially in semi-active control.

This contribution continues our earlier work on RL-based control of a seismically excited shear-type building, which examined the robustness of a typical single-agent RL controller. Here, the investigation is extended to multi-agent RL control, in which the ultimately applied control action is determined based on the recommendations of a group of independent RL agents. Control is applied via a tuned mass damper whose mass can be locked to or unlocked from the story in an on/off manner. The reward function used for agent training quantifies structural vibrations, and additionally, it includes a penalty term that promotes locking at moments of zero relative velocity, in order to avoid the associated impacts.

Keywords:
Tuned Mass Damper, Semi-active Control, Reinforcement Learning, Multi-agent Control, Robustness

Abstract:
This contribution presents a semi-active structural control strategy built on reinforcement learning (RL). Even though RL has recently achieved superhuman performance in a range of challenging, control-oriented real-world tasks, its use in structural control remains limited. A seismically excited shear-type structure is considered and fitted with a tuned mass damper (TMD). This TMD, after optimal tuning, is used as a baseline for comparison. The proposed RL control system switches the damping coefficient of the TMD dashpot between two discrete levels corresponding to a negligible damping and a high damping state. The switching policy is encoded in the form of an artificial neural network that is trained through a simulated trial-and-error process. Sensor measurements serve as inputs to the network. The control signal is the outputs from the network. The training process seeks to maximize control performance in terms of interstory drift and total mechanical energy reduction. Abrupt activation of viscous damping can result in inelastic impacts, an additional acceleration-based penalty term is used to limit such effects. Results show that the RL-based controller outperforms the optimally tuned passive baseline and has reasonable tolerance to measurement errors. The degree of this robustness depends on the number of sensors.
The authors gratefully acknowledge the support of the National Science Centre, Poland, granted under the grant agreement 2020/39/B/ST8/02615. For the purpose of Open Access, the authors have applied a CC-BY public copyright licence to any Author Accepted Manuscript (AAM) version arising from this submission.

Keywords:
semi-active control, structural control, reinforcement learning, bang-bang control

Abstract:
This contribution discusses a semi-active control technique intended for mitigation of structural vibrations. The control law is implemented using the machine learning technique of reinforcement learning, that is in a repeated trial-and-error interaction between the control agent and a simulated environment. Such an approach allows to omit the stage of deriving the optimal control in an analytic way, which is often difficult in nonlinear, semi-actively controlled systems. A specific implementation of the Deep Q Learning (DQN) approach is applied, which promotes control robustness with respect to structural damages. A dedicated network architecture allows the network to be damage-aware, and a specific training procedure involves not only the observations, control actions, and rewards, but also the current health status of the structure.

A numerical example is provided involving a shear-type building model subjected to a seismic excitation. The actuator takes the form of a tuned mass damper (TMD), which is semi-actively controlled by changing the level of viscous damping. The optimally tuned classical passive TMD is used as the baseline reference damping system.

Abstract:
This presentation considers a semi-active control method aimed at the reduction of structural vibrations in the presence of unknown structural damages. The control algorithm is developed using reinforcement learning [1], a machine learning technique characterized by an iterative trial-and-error interaction of the control agent with the controlled structure. A quasi-optimal control law is derived by observing and learning from the collected interaction experience. By being data-driven, this strategy bypasses the need for an analytical derivation of optimal control, which can be challenging in semi-active and nonlinear control systems [2]. The approach of double Deep Q Learning (DQN) with experience replay is used. It builds upon earlier results [3], but here the aim here is to promote control robustness in the presence of unknown structural damages. The control algorithm is ultimately encoded in the form of a trained artificial neural network with a custom architecture that involves a dedicated damage-identification branch.
The effectiveness of the approach is demonstrated using a numerical model of a structure subjected to a seismic-type random excitation. A semi-active tuned mass damper (TMD) is employed as the actuator, and the control signal affects its viscous damping properties. The reference baseline is provided by the optimally tuned, classical passive TMD.

Keywords:
Structural control, Semi-active control, Structural health monitoring (SHM), Reinforcement learning, Machine learning

Abstract:
This contribution reviews a recently proposed control strategy for mitigation of vibrations based on the Prestress-Accumulation Release (PAR) approach [1]. The control is executed by means of semi-actively controllable truss-frame nodes. Such nodes have an on/off ability to transfer bending moments: they are able to temporary switch their operational characteristics between the truss-like and the frame-like behaviors. The focus is not on local energy dissipation in the nodes treated as friction dampers, but rather on stimulating the global transfer of vibration energy to high-order modes. Such modes are high-frequency and thus highly dissipative by means of the standard mechanisms of material damping. The transfer is triggered by temporary switches to the truss-like state performed at the moments of a high local bending strain. A sudden removal of a kinematic constraint releases the locally accumulated strain energy into high-frequency and quickly damped vibrations.
The first formulation investigated global control laws [1]. Recent approaches generalized it to decen-tralized control with a local-only feedback, which was tested in damping of free vibrations [2] as well as forced vibrations [3]. Recently, a global formulation was proposed that aims at a targeted energy transfer between specific vibration modes [4], and attempts were made to go beyond skeletal struc-tures [5]. Numerical and experimental results will be presented to confirm the high effectiveness of the approach in mitigation of free, forced random and forced harmonic vibrations.

Abstract:
This contribution reviews a recently proposed semi-active control approach based on the Prestress-Accumulation Release strategy, which aims at damping of structural vibrations by promoting vibration energy transfer from lower- into higher-order modes that have significant material damping. Unlike typical semi-active control, which focuses on local dissipation in actuators, the aim is to trigger natural global damping mechanisms. The actuators are controllable truss-frame nodes: lockable hinges that can change their mode of operation from a frame node (locked hinge) into truss node (free rotation). Sudden removal of such a kinematic constraint releases the accumulated bending energy into high-frequency quickly damped local vibrations. Two formulations are reviewed: decentralized with local-only feedback, and global, which aims at a targeted energy transfer between specific modes. Experimental results confirm the effectiveness using free, forced harmonic and random vibrations.